Tiger Team Meeting : Parser As A Service (16th January 2018)

Background :

• Proposal for Release 2 / Beijing from Vodafone on Centralization of Parser / Parser As A Service
  • Various projects in ONAP are using different Parsers for either model design (en-coding) or model de-coding 
  • The same function is being performed at many places and there is no consistency in Parser Approach across projects
  • Vodafone's proposal is to centralize the Parser as part of Common Services Layer or Modelling Project and expose that As-A-Service
  • This will bring consistency in Parsing function within ONAP and paves the way for an ultimate model driven stack
  • This also opens up the platform implementation approach for Parsers and provide a placeholder within ONAP for a "Parser Surgery"
  • This common placeholder can be used to on-board "Multiple Parsers" while retaining the parser exposure to consumers (design/runtime components or even external components in future) via an abstracted "unified / common" API
•  Another proposal from ZTE on the same concept, "Parser As A Service" was received for R2 delivery, with OPNFV parser for VFC Component
• It is a similar parser idea, which was received from ZTE along with another proposal for a centralized / platform based  Run Time Catalogue by ZTE Central RTC has been "tentatively" approved as a new project
• ZTE has given more details of the proposed architecture and how the common / central parser as a service would look like
• SDC team presented with an idea of exposing SDC parser as a service, proposal was to do a pilot / PoC in R2 for exposing SDC Parser As A Service from SDC project itself

Proposal As Part of 9th January 2018 Modelling Subcommitte Call : Atul.Purohit1@vodafone.com to conduct a series of calls with different stakeholders to identify the right approach and put up a consolidated proposal for R2, potentially going into R3 via a PoC / trial / adoption route and bring to modelling sub-committe for consideration. The same was noted here : Modeling sub-committee meetings#committeemeetings-20180109MeetingAgenda&Minutes

Meeting / Call notes from 16th Janury 2018 :

Attendees : denglingli@chinamobile.com (China Mobile), s.shenbaga.rajan@ericsson.com (E///), denghui12@huawei.com (Huawei), zhang.maopeng1@zte.com.cn (ZTE), soareluna@gmail.com (E///), rittwik.jana@gmail.com (AT&T), shang.xiaodong@zte.com.cn (ZTE), Atul.Purohit1@Vodafone.com (Vodafone)

Could Not Join : ml636r@intl.att.com (AT&T)

For Information : Present Release 2 Approved Requirements : Beijing Functional Requirements

Current Parser Distribution in ONAP :

Points Discussed :

1. Defined problem statement to have a common understanding among teams
2. General Agreement was that there is a need to have a common / central parser placeholder within ONAP, "definition of common" is where more than one component uses the same Parser Implementation
3. Use of TOSCA happens at two places, (a) For defining the service models in a declarative fashion and (b) Within components for orchestration in a declarative fashion
4. In an ideal world if the service models were common and every component could do orchestration and fulfilment in a declarative fashion, there would be no need to have two levels of implementations
5. However TOSCA is so generic that there is a need to have custom changes / extensions in TOSCA by individual implementations to extract full set of functionality by the implementing component
6. Stateless approach is not easily feasible in TOSCA context
7. Core capabilities of the parser(s) themselves, they are not equally capable and hence the need to have extensions at individual component level (Core Capabilities of Parser comes from if the parser is based on Sample YAML profile 1.0 or 1.1 or 1.2)
8. While everyone agrees that a central "Parser As a Service" is a great idea, but what should be the extent of central exposed capability has to be decided (along with a governance model) and when it comes to consumers of this capability, should the consuming systems negotiate that capability and still apply extensions which are local in nature to the component itself ? or should everytime an implementation component proposed extensions, they should be subsumed in central Parser Placeholder and be exposed as a service
9. Key theme / learnings from UUI tasca parser implementation is "Exception Handling",
   for example,
   Some parsers can parse only sample yaml profile 1.1 or 1.0 or both versions,
   Depending on client (the component which is implementing the parser capability) there is a need to discuss and decide an Error/Exception report (format), Errors/Exceptions can occur at different phases - some components (execution platforms) stops the parsing at the very first step, some execution platforms (components) do not take automatic corrections because they do not contain the overall service intent
   If we build compatible representation of output VID - run time view of what is running where, app-c has run time visibility, four components have different UI Unification story in SDC JASON error report, standardize representation not the outut How far are you going to parse if there is an exception any mix of 1.0 or 1.1 document, build extentions and mix them in implementation (Serben to share his UUI implementation experience / challenges / learnings)
10. So far, three TOSCA based parsers are submitted to "Modelling Project", each having a variation in Parser Defintion and the API to access them is also different, details are here Modeling API

Excerpt of UUI / javatoscachecker implementation experice (Serben)

      What is the Scope of "Parser As A Service" ?

• Does the Parser Contain Well Formed YAML
• Is it Syntactically Correct TOSCA (Grammer to be checked against TOSCA Specifications)
• Coherence Checking (As Much As Possible) : Type Reference / Property Reference etc...

      What is the input to Parser ?

• Usually a CSAR Archive File

Questions :

Is the CSAR File Self Contained ?

Are all documents within it, or does the TOSCA parser requires some additional details ? (If we agree that Parser As A Service function is Stateless, there is a need for CSAR to be self-contained)

If CSAR is a single file, does it contain all necessary specifications ?

If there is multi-part content representation, should it be with one document or part of it ?

Observation : The tool should be able to resolve the whole import tree i.e. resolve all dependencies.

      What is the expected output / response of this "Parser As A Service" function ?

• For the consumer of the "Parser As Service" function, the first response is a simple - Yes (Valid Call) or No (In-Valid Call)
• If the response is "Yes" move to next step in "Parser As A Service" Call, if Not, Send back an exception / error code
• If answer is No, list down all possible exceptions (error codes) that can be sent back to the requestor with some additional text for the receiver to take corrective action and re-trigger the "Parser As A Service" call
• As Part of R2 PoC, trial / design activity, define all exceptions from all parsers and standardize on the API specifications for call, exceptions, request - response etc...
• If the response of the "Parser As A Service" function is Yes Valid Request, the next stage is an Output TOSCA Object Model, (there is a need to standardize both inputs and output representations here)
Observation : If the transactions / interactions are REST representable, and are stateless, that means that every transaction should have every possible information on there, which may not be practical (how to overcome that)

      Requirements Gap / Observations from UUI javatoscachecker implementation in R1

      Most TOSCA users separate schema (type definitions) and templates. For one schema the users have a large(r) set of templates to validate. The users do not want to submit the schema every time (or re-assemmble a csar every time) so   

      we had to provide a stateful api.

      Solution in the checker:

       - The user creates an application 'space' to which the server associates all type definitions. Because of the imports structure the API allows a client to submit 'named' documents (the processor's output will become part of the state and the

         document can be referenced) and 'anonymous' documents (the output is reported back to the client but does not become part of the state)

       - Extensions: some clients have extensions to the standard which require a new grammar and optionally additional coherence checks. The service needs to account for all this variations. This is also true for different versions of the spec.

       - The output of this service will be (in most cases) processed by some piece of software. While the REST API might not changed the set of errors and backwards compatibilty of the response representation are an issue as the underlying

         processing evolves.

Action Points for Next Call : On 22nd January 2018

1. Atul to consolidate requirements, suggest a tentative plan of action for R2 and R3 on "Parser As A Service" Project
2. Create a Proposed Architecture Deck for Modelling Sub-Committee Presentation on 23rd January 2018
3. Secure an Agreement for Proposed Plan
4. Agree / Propose Set of Details Architecture / Design Artefacts Required for R2 & R3
5. Agree with TSC / Line Up Calls with various PTLs on "Parser As A Service" concept and agree a buy in for implementation