Executive Summary
General objectives of the ONAP Common Versioning Strategy (CVS) for all APIs:
- Implement semantic versioning (MAJOR.MINOR.PATCH) for APIs
- If necessary, refactor APIs to support the concept of MINOR releases; versioning scope and use cases provided
- Adopt a BWC policy for APIs that is current MAJOR release minus 1 year (to be re-visited post-Casablanca)
HTTP/REST API specific rules/policies:
- Implement pre-defined custom headers to communicate MINOR, PATCH, and LATEST VERSION
- Align URLs to include only the MAJOR version
- Documentation using Swagger 2.0, by following the guidance provided
ONAP CVS Proposal Deck was presented to the Architecture Committee on 4/3/2018, and modified on 4/11/2018 to correct the flow diagrams and case of the custom header names.
Clarification on terminology:
- SHALL (mandatory) is used to indicate a requirement that is contractually binding, meaning it must be implemented, and its implementation verified.
- SHOULD (non-mandatory) is used to indicate a goal which must be addressed by the design team, but is not formally verified.
Policies for All APIs
Implementing Semantic Versioning for APIs
- Utilizes the same semantic versioning methodology that is being used for ONAP’s Release Versioning Strategy; therefore, development teams are familiar with the definition of the methodology.
- For a given a version number, MAJOR.MINOR.PATCH, increment the:
- MAJOR position when you make any incompatible API change
- MINOR position when you add functionality in a backwards-compatible manner
- PATCH (or BUILD) position when you make invisible (and thus backwards-compatible) bug fixes
- Details of the specification can be found at http://semver.org/
Versioning Scope and Use Cases
Definition of Layers (second column in the table below)
- API-wide - versioning at this level creates a new set of URIs to identify the resources
- Resource-wide - independently version different parts of the API's resource model; major change to a specific resource model with the notion that not all the resource models will change (namespace changes, add a new resource)
- Representation layer - modify and enhance the format, structure or content of resources that are served by the API without changing the resource model
- Behavior change - changes in behavior that do not change the resource model or representation; can include changes in process or state management
Use Case | Layer | MAJOR | MINOR | PATCH1 |
---|---|---|---|---|
Refactoring resource model, if it has become fragile or overly complex through many evolutionary steps; introduce a set of namespaces to reflect the category of resources (nothing is where it used to be) | API | X | ||
Restructure resource(s) to meet new business requirements/conform to emerging interface standard(s) | API | X | ||
Add a required request parameter value without default | API | X | ||
Add a required request parameter value with default | API | X | ||
Add a new resource model or type | Resource | X | ||
Update an existing resource model or type w/BWC | Resource | X | ||
Update an existing resource model or type w/out BWC | Resource | X | ||
Adding optional data items to an input resource representation | Resource | X | ||
Add a new behavior method w/no changes to existing behavior methods (e.g. add PUT as a method when it did not exist as prior functionality) | Behavior | X | ||
Changes to data volume on returned response | Behavior | X | ||
Changes to undocumented sorting order of data on returned response w/out changes to volume | Behavior | X | ||
Changes to documented sorting order of data on returned response w/out changes to volume | Behavior | X | ||
Changes in behavior that do not change the resource model or representation | Behavior | X | ||
Deprecate method, but do not change the structure of the resource model or representation | Behavior | X | ||
Adding new optional parameter(s) (that do not change default behavior) to requests | Representation | X | ||
Adding data items to an output resource representation, where any prescribed schema validation (for example, XML Schema or JSON-Schema validation) is not broken | Representation | X | ||
Fix a defect that does not impact behavior or representation (e.g. fix internal algorithm to run more efficiently) | General | X | ||
Changes to error codes, whereas the error code content is updated or changed, with no change in the resource model or representation | API | X |
1 PATCH refers to the position in the version number, not the HTTP method of PATCH. This method should not be used as it is idempotent.
Backward Compatibility (BWC) Policy - TO BE REVISITED DURING OR POST-CASABLANCA
- API BWC shall be defined for MAJOR releases as the current release - 1 year (to be re-visited post-Casablanca). In other words, if an API is currently at 1.12 and a MAJOR release occurs to increment the version to 2.0, 1.12 (which is BWC for versions 1.0-1.11) must be functional/available for the period of 1 year after 2.0 is released.
- API owners shall ensure the previous MAJOR release remains available and functioning, in its last available production state, for the period of the BWC policy.
- MINOR releases shall be not time or release-based, as they are assumed to be BWC.
- API owners shall ensure no end-to-end services break with the deprecation of an API, due to the BWC Policy. End-to-end services includes, but is not limited to, VNFs, PNFs, Networks, Allotted Resources, etc.
Policies for HTTP/REST APIs
API Custom Headers and Behavior
Custom Headers and General Rules
- Three custom headers shall support versioning in APIs:
- X-MinorVersion
- X-PatchVersion
- X-LatestVersion
- The request from the client shall not break, if the headers are absent in the request.
- Clients shall ignore additional values from the payload in the response, if provided.
- It shall be explicitly specified in the interface contract that a server may increment a MINOR version and add additional fields.
- The client shall be capable of handling this type of change in contract, if they remain on a previous MINOR version.
- The server shall employ logic to fallback to the MAJOR version of the API, in the event that X-MinorVersion is not provided (see use case below).
Use case for falling back versus failing forward to the MAJOR version of the API
The vserver entity in v1 of ECOMP had no “prov-status” field.
The prov-status field was added in v1.1 as a non breaking change.
The RO client PUTs all the data in the vserver except the prov-status field, so they use v1.
The GFP client manages the prov-status field in the vserver. They use v1.1.
A REST PUT must include the entire representation of the object.
Therefore a v1 PUT does not include the prov-status but the v1.1 PUT must.
If only major versions get passed, and the system should fail forward, a PUT by RO lacking the prov-status field would wipe out the prov-status value.
Custom Headers Specification
Header Name | Specification |
---|---|
X-MinorVersion |
|
X-PatchVersion |
|
X-LatestVersion |
|
Custom Header Flow Diagrams
URL Structure Policy
- The URL shall only contain the MAJOR version number to minimize changes in the URL for MINOR and PATCH releases, assuming MINOR.PATCH releases are BWC.
- The structure of the URL shall be as follows, where version is placed after the "service" or API name:
…/root/{service or API name}/v{version number}/{resource path}
Example: {hostname}/aai/resource/v14/complexes
*Note: "v" should precede the MAJOR version number in the URL. Service or API name is not the resource; it is intended to group of set of related resources.
FOR RESTCONF APIs ONLY
In RESTCONF, APIs are organized by "modules", which for our purposes we can say are analogous to services. There are 3 different types of APIs, each with its own standard URI format:
- Configuration data (also referred to as "config tree"), which stores the in flight view of network data (so it shows pending changes).
- Required URI is /restconf/config/{module}/{resource}
- Operational data (also referred to as the operational tree), which stores the current active view of network data.
- Required URI is /restconf/operational/{module}/{resource}
- RPCs
- Required URI is /restconf/operational/{module}/{rpc name}
URIs for ONAP will follow the convention below:
- Configuration data: /restconf/config/{service}:v{version}_{resource} e.g. /restconf/config/neutron:v2_networks
- Operational tree : /restconf/operational/{service}:v{version}_{resource} e.g. /restconf/operational/neutron:v2_networks
- RPC : /restconf/operations/{service}:v{version}_{rpc} e.g. /restconf/operations/SLI-API:v1_healthcheck
Requirements/Extensions of the Swagger 2.0 Specification
Spec ID | Specification/Requirement |
---|---|
SG-1 | All components shall use Swagger 2.0. The specification may be found here. OpenAPI 3.0 is a roadmap item. |
SG-2 | Within the Info Object, the following annotations are included in the Swagger specification and shall be required, even if they are optional in the Swagger spec: title description version - fully-qualified version number of the Swagger file (ex: 1.4.18) |
SG-3 | Within the Info Object, the following are extensions of the Swagger specification and shall be required: x-planned-retirement-date - use YYMM; string type. This is the date that the API shall be deprecated, based on the BWC Policy. NOTE: APIs may be active after their retirement date, but are not guaranteed to remain in production. An API retirement may be pushed out to accommodate BWC for clients. x-component - SDC, MSO, SNIRO, etc., or the mS name; string type. This is the component that primarily owns the API from a development perspective. |
SG-4 | Under the Path, the following shall be required: x-interface info - this contains two attributes:
|
SG-5 | Swagger files shall be generated at build time, and be placed in a centralized ReadTheDocs repository: http://docs.onap.org |
SG-6 | Within the Path Item Object, the following are included in the Swagger specification and shall be required: description - string parameters
|
Working Team Information/Discussion
Working Team Members
Name | Company | Phone Number | Time Zone | |
---|---|---|---|---|
AT&T | ||||
AT&T | dw2049@att.com | (561) 371-7619 | EST | |
AMDOCS | sharon.chisholm@amdocs.com | EST | ||
Huawei | Christopher.Donley@huawei.com | |||
AT&T | gg2147@att.com | (847) 420-8459 | ||
Mark Ho | AT&T | mh574f@att.com | (781) 791-4345 | EST |
VMware | ramkik@vmware.com | |||
AT&T | am803u@att.com | EST | ||
ARM (OAM) | adolfo.perez-duran@oamtechnologies.com | 720.560.2659 | MT | |
Intel | alex.vul@intel.com | |||
Huawei | Parviz.Yegani@huawei.com | (408) 759-1973 | PT | |
ZTE | mailto:zhao.huabing@zte.com.cn | GMT+8 |
* Responsible team lead
Discussion Items
Item | What | Notes |
---|---|---|
1 | R3 Focus/Scope | Establish/finalize a proposal for a generic versioning methodology, URL structure for HTTP/REST APIs, and Swagger 2.0/OpenAPI 3.0 guidance.
Current recommendations (on the table):
|
2 | R4 and Beyond | Establish/finalize a proposal for backwards compatibility (BWC) and exposing API versions.
|
3 | Noteworthy | Dana Bobko will be working with Gregory Glover to combine the results of this working team with the Documentation effort (there are intersection points). |
4 | Open issue | How do we deal with restconf interfaces (e.g. ODL-based components)? See Dan's comment |
Resources/Related Links
- REST APIs Must be Hypertext-Driven: Blog post where Roy Fielding argues that not any HTTP-based interface is a REST API
- RESTful API Design Specification (for ONAP)
- REST APIs don’t need a versioning strategy – they need a change strategy
23 Comments
user-9823d
This is a great proposal, I just have two comments:
1. The header version negotiation mechanism seems complicated. If we want to achieve something like that, the ONAP service implementations need to maintain a lot of states about the version, like which object fields are added at which minor version. Given the minor version is just some changes which are supposed to be back-compatible, should we just use the URL to indicate the MAJOR version and always send the latest MINOR and PATCH version of that MAJOR version to the client?
2. Can we use the url structure in the RESTful API Design Specification ?, it's already widely adopted in ONAP and is aligned with the version strategy proposing here. Reusing it can avoid a lot of unnecessary rework.
Dana Bobko
1. We can’t fall forward because of the REST implications. For example:
If v1 of a payload has: {
Field1
Field2
}
And v1.1 of a payload has
{
Field1
Field2
Field3
}
Falling forward would not give the server enough knowledge about whether:
the client knew about 1.1 and just didn’t send the third field in their payload, or
if the client did NOT know, speaks 1.0, and only knows to send the two fields.
The PATCH version is only being set back to indicate to the client that something did change, in the event that the server introduced what it thought was a MINOR change.
2. How is the URL structure in the Spec you provided different from what is proposed here? I provided a real world example above for you to use as a reference. The Spec talks about dogs and pet stores so it is hard for me to tell.
user-9823d
1. I think if it's a MINOR change, it is supposed to be back-compatible, so the client can handle it without the notion of the changed part, otherwise, it's a MAJOR change.
2. The suggested URL structure is http://[host]:[port]/api/{service name}]/v{version number}/{resource} , if we need a concrete example, we can use http://{hostname}/api/multicloud/v0/vim_types
Alexander Vul
I am not sure I buy the example above. I would imagine that with versioning in place, the communication between two functions would take place iff both functions can make use of the same interface version. Mismatches of interface versions during communication would not be allowed....
user-9823d
I agree that "both functions can make use of the same interface version", however, they only need to align with each other on the MAJOR version. According to Semantic Versioning:
When I think about "back-compatible", my assumption is that a lower version client can talk with a higher version server or vice versa. So only the MAJOR version should be taken into consideration when the client and server are talking with each other, which has already been handled by the "version" part of the URL.
Alexander Vul
Huabing,
I've asked this question before, I am goint to ask it again... Why do we need the port number exposed in the URL? We can either establish a convention for how ports are used my microservices, or map incoming API calls to specific ports as part of the registry lookups.
What do you think?
Alex
user-9823d
Hi Alex,
It's an "HTTP URL format standard", so the port should be there, otherwise, the request might not be sent out because the SDK/framework(let's say, the java http lib) you are using assumes the port is 80, which is not true in most cases.
Dan Timoney
I posted a comment last year on the wiki page the url structure in the RESTful API Design Specification indicating technical issues with the proposed URL format. Here is that comment again :
The URI format above does not fit well for controllers based on OpenDaylight - or more generally, for controllers that expose RESTCONF interfaces. There are existing conventions in RESTCONF that are built in to OpenDaylight (and probably other controller platforms as well) that could make it difficult to meet the URI standards above exactly, but we can get fairly close.
In RESTCONF, APIs are organized by "modules", which for our purposes we can say are analogous to services. There are 3 different types of APIs, each with its own standard URI format:
I propose the following conventions for versioned RESTCONF APIs for ONAP:
This is fairly close to the proposed standard, with the exception that the separator between version and resource is an underbar instead of a slash.
If this is not acceptable, then SDNC and APPC will need to change their architecture to route through some form of proxy (e.g. apache) whose only purpose is to rewrite the URL, which seems inefficient and error prone.
Dana Bobko
I think this proposal would work. I will add it to the agenda when we meet next on 6/6/18.
Jack Lucas
There's an alternative approach to versioning REST-ish APIs that relies on standard features of HTTP rather than new, proprietary headers, and that provides for more granular versioning. Specifically, the alternative approach versions the individual resource representations rather than the entire API. There's a concise discussion of this approach at http://barelyenough.org/blog/2008/05/versioning-rest-web-services/. The approach proposed here actually makes it difficult to implement the hypertext-driven APIs suggested in the Fielding reference above (reference #1); the alternative is more consistent with Fielding's approach.
Was this alternative considered?
Pamela Dragosh
This is a very simple, sensible way of making sure the payloads are compatible. Much easier to implement and enforce than the proposal above.
Dana Bobko
I can send you the meeting invite to the next Versioning Subcommittee meeting so you can voice your concerns.
Dana Bobko
Several approaches were discussed, including this one.
I can send you the meeting invite to the next Versioning Subcommittee meeting so you can voice your concerns.
Jack Lucas
Here's another description of the alternative approach: https://apigee.com/about/blog/developer/common-misconceptions-about-api-versioning
Dana Bobko
This link was shared with the working team during our discussions.
Pamela Dragosh
Chris Donley - this proposal seems like a lot of overhead. What problem is it really trying to solve? I am going to request that the PTL's be allowed to vote as to whether to adopt this or not. Since we are the ones that have to implement it.
Dana Bobko
I can send you the meeting invite to the next Versioning Subcommittee meeting so you can voice your concerns.
This proposal was presented to the PTLs a few weeks ago (I'm sorry I don't recall the exact date). It was combined with the documentation proposal put forth by Gregory Glover.
Michael Hwang
I want to confirm - is the proposal suggesting to have a single version for a given API for all its resources? Or a version for the API level and separate versions for the individual resources?
If its the former, then that means for an API with many resources that a change to a single resource would affect the version of all the other resources correct?
Dana Bobko
In the use cases, adding a resource would increment the minor version of the API.
Michael Hwang
So if my API is on version 1.0.0 and I have:
And I add a new resource called
bar
that does not impactfoo
. Is the following correct?:My question is whether the version is global, on the individual resources, or both?
If the version is on the resources then
X-LatestVersion
wouldn't change in the example above.Dana Bobko
You are correct. The URL would not change for bar. If the client wanted to access bar, they would pass in the X-MinorVersion=1 in the header to access that resource.
The version is at the namespace level, not the resource level.
Keong Lim
Just reiterating a comment on another page RESTful API Design Specification with a counterpoint to this method of versioning:
http://blog.steveklabnik.com/posts/2011-07-03-nobody-understands-rest-or-http
See also discussion at StackOverflow: https://stackoverflow.com/questions/972226/how-to-version-rest-uris
Dana Bobko
We debated this option on the working team. The URL structure is just one factor of the overall strategy. The challenge with versioning APIs is that there are several ways to deal with it, and there is no "right" or "wrong" way to do it. The working team had to decide on one.
You're welcome to propose a complete alternate strategy and bring it to the ONAP CVS team; however, this API strategy has already been slated for Casablanca. It was socialized with the TSC and PTLs on numerous occasions, as well as the Architecture Subcommittee.
Here is the link to the CVS working team's page with the meeting information: Common Versioning Architecture Team