Project Recommendations for Package Upgrades
AAI R3 Security/Vulnerability Threat Analysis
Faster XML Jackson usage in Portal Code and replacing it with Gson
15th July 2019: "Out of Scope" as per Fixing Vulnerabilities in the ONAP Code Base
Security subcommittee has recommended teams move away from jackson, and will be presenting alternatives and asking for an assessment from each project. Our team will need to do an analysis - this would not be trivial, especially given how many of our repos are impacted. As of now, this would be a very high LOE for the team, we need to understand what the recommendation from the SECCOM is before we can provide better details on what the LOE would be.
Articles with comparisons and benchmarks:
Rationale for eliminating some options from the articles above (about 20 libraries in total):
Short-list of libraries as reasonable options to be explored, including:
Quick CVE comparison:
Search on AAI source code shows:
import com.fasterxml.jackson.databind.JsonNode; import com.fasterxml.jackson.databind.ObjectMapper; public synchronized void loadUsers(String authFilename) throws Exception { users = new HashMap<>(); mapper = new ObjectMapper(); // can reuse, share globally JsonNode rootNode = mapper.readTree(new File(authFilename)); JsonNode rolesNode = rootNode.path(AuthConstants.ROLES_NODE_PATH); for (JsonNode roleNode : rolesNode) { String roleName = roleNode.path(AuthConstants.ROLE_NAME_PATH).asText(); AuthRole role = new AuthRole(); JsonNode usersNode = roleNode.path(AuthConstants.USERS_NODE_PATH); JsonNode functionsNode = roleNode.path(AuthConstants.FUNCTIONS_NODE_PATH); for (JsonNode functionNode : functionsNode) { String function = functionNode.path(AuthConstants.FUNCTION_NAME_PATH).asText(); JsonNode methodsNode = functionNode.path(AuthConstants.METHODS_NODE_PATH); boolean hasMethods = handleMethodNode(methodsNode, role, function); if (!hasMethods) { // iterate the list from HTTP_METHODS for (HTTP_METHODS meth : HTTP_METHODS.values()) { String thisFunction = meth.toString() + ":" + function; role.addAllowedFunction(thisFunction); } } } handleUserNode(usersNode, roleName, role); } usersInitialized = true; } |
import com.fasterxml.jackson.core.JsonProcessingException; import com.google.gson.JsonArray; import com.google.gson.JsonElement; import com.google.gson.JsonObject; import com.google.gson.JsonParser; private synchronized void reloadUsers() { Map<String, AAIUser> tempUsers = new HashMap<>(); try { LOGGER.debug("Reading from " + globalAuthFileName); String authFile = new String(Files.readAllBytes(Paths.get(globalAuthFileName))); JsonParser parser = new JsonParser(); JsonObject authObject = parser.parse(authFile).getAsJsonObject(); if (authObject.has("roles")) { JsonArray roles = authObject.getAsJsonArray("roles"); for (JsonElement role : roles) { if (role.isJsonObject()) { JsonObject roleObject = role.getAsJsonObject(); String roleName = roleObject.get("name").getAsString(); Map<String, Boolean> usrs = this.getUsernamesFromRole(roleObject); List<String> aaiFunctions = this.getAAIFunctions(roleObject); usrs.forEach((key, value) -> { final AAIUser au = tempUsers.getOrDefault(key, new AAIUser(key, value)); au.addRole(roleName); aaiFunctions.forEach(f -> { List<String> httpMethods = this.getRoleHttpMethods(f, roleObject); httpMethods.forEach(hm -> au.setUserAccess(f, hm)); this.validFunctions.add(f); }); tempUsers.put(key, au); }); } } if (!tempUsers.isEmpty()) { users = tempUsers; } } } catch (FileNotFoundException e) { ErrorLogHelper.logError("AAI_4001", globalAuthFileName + ". Exception: " + e); } catch (JsonProcessingException e) { ErrorLogHelper.logError("AAI_4001", globalAuthFileName + ". Not valid JSON: " + e); } catch (Exception e) { ErrorLogHelper.logError("AAI_4001", globalAuthFileName + ". Exception caught: " + e); } } |
FasterXML Jackson version | Google gson version | Comments | ||
---|---|---|---|---|
|
| |||
|
| Jackson's JsonNode is a more abstract data structure, compared with Gson's more concrete data structures JsonObject and JsonArray. | ||
|
| Code structure differs at this point (function name vs role name) but the general intent of the code is equivalent (get the element name as a string). | ||
|
| For some reason, this version still catches com.fasterxml.jackson.core.JsonProcessingException even though it uses Google gson for parsing. Not a good idea to defer exception handling to the caller since the caller has no idea why/how/when/where the parsing failed and might be left with an invalid data structure as well. | ||
|
| Method call vs Java lambda call is not really relevant to the Jackson replacement, but consistency of style could be an overall goal if the code is being re-factored anyway. |
I have investigated the feasibility of replacing all Jackson Spring Boot dependencies with Gson, by converting the two AAF security microservices, aaf-fproxy and aaf-rproxy to use Gson only.
The basic method I followed is detailed here: https://www.callicoder.com/configuring-spring-boot-to-use-gson-instead-of-jackson/
What I did:
At this point, if the application is only using Jackson for automatically serializing and deserializing request and response objects in its REST APIs, the conversion should be complete. Your Spring Boot application should now be switched to using the Gson implementation, and function as before.
Notes:
Example an example of the changes I made to the pom.xml can be found in this changelist for the AAF rProxy project: https://gerrit.onap.org/r/gitweb?p=aaf/cadi.git;a=commitdiff;h=0d9b3896ad594816b1eb7048949114e6a18c4bd4
(Note that this changelist contains other code changes but only the pom.xml changes are required for switching to Gson)
Actions:
From: Greg Matza [mailto:greg@scylladb.com]
Sent: Friday, 12 October 2018 09:56
To: Keong Lim <Keong.Lim@huawei.com>
Subject: ScyllaDB
Keong,
I saw your post on the Cassandra mailing list.
ScyllaDB is an open source drop-in replacement for Apache Cassandra. You can easily substitute Scylla for Cassandra in JanusGraph - in fact IBM has publicly spoken about why they chose Scylla instead of Cassandra for their JanusGraph as a Service. https://www.scylladb.com/tech-talk/performance-evaluation-scylla-database-backend-janusgraph-scylla-summit-2017/
And, because ScyllaDB is written in C++, there are no Jackson dependencies. So you'll be able to satisfy your security concerns, and take advantage of the fact that Scylla is faster and easier to manage.
We believe that there are already users of ScyllaDB within Huawei. For example, in March of 2018, our CEO and CTO met with Huawei Cloud, who are interested in offering Scylla as a Service. That potential partnership is still under discussion.
If you are interested in learning more, I would recommend dowloading the software at https://www.scylladb.com/download/, and joining our ScyllaDB Users Slack, where there are many Scylla users and engineers who can answer questions. http://slack.scylladb.com. (There is even a Chinese-language channel - #general-cn - on the Slack, if you prefer that to the mostly-English #general channel)
Good luck, and I hope to see you on the Slack!!!
Greg
---
Greg Matza
650-400-9648
From: Greg Matza [mailto:greg@scylladb.com]
Sent: Friday, 12 October 2018 12:02
To: Keong Lim <Keong.Lim@huawei.com>
Subject: Re: ScyllaDB
Sounds great!
I know that time is very short, but we are holding the Scylla Summit in 4 weeks - 6th November and 7th November. We will be near San Francisco. http://www.scylladb.com/scylla-summit-2018/ We welcome you and your team to attend the Summit, in order to better understand the technology and integrate with the user community.
There is also a full day of training on 5th November that would likely be valuable.
If it is possible for you and your team to come, I will be happy to waive the $500 fee for Training and Summit. Let me know, and I can arrange for the free registration. If necessary, I can also get you a letter of invitation, which may help with obtaining a visa.
Greg
From: Greg Matza [mailto:greg@scylladb.com]
Sent: Wednesday, 24 October 2018 03:58
To: Keong Lim <Keong.Lim@huawei.com>
Subject: Re: ScyllaDB
Keong,
Google Alert notified me of your posting on the ONAP site. From that notification, I saw your notes on our conversation and learned a bit more about your project.
We are excited by the prospect of inclusion in the ONAP AAI project. Our co-founders - Dor Laor and Avi Kivity are well experienced with Linux Foundation projects, as they are the creator and early manager of the KVM Hypervisor. So working with OSS Foundations is in our company's DNA. In addition, telecom are among the early adopters of Scylla - AT&T, Verizon, Huawei, Comcast, T-mobile are all either running Scylla in Production or are in POC.
As you continue to research the viability of Scylla for AAI, I'd like to make our technical and executive resources available to you. From a technical perspective, we are available to help with any POC or evaluation. (Installation, hardware recommendations, monitoring setup, or other practical questions). From an executive perspective, we'd be happy to discuss any potential questions with roadmap, licensing or other 'vision'-type questions.
Let me know if/when you are ready to engage with our technical or executive resources.
Greg
From: Greg Matza [mailto:greg@scylladb.com]
Sent: Tuesday, 11 December 2018 13:01
To: Keong Lim <Keong.Lim@huawei.com>
Cc: stephen.terrill@ericsson.com; jf2512@att.com; Maheedhar Gunturu <maheedhar@scylladb.com>
Subject: Re: ScyllaDB
Keong,
Unfortunately, we don't have a public demo lab that you can point your application at. The closest we have is the Scylla Test Drive, which spins up a cluster on AWS and gives you SSH access to the nodes, but the cluster only runs for an hour, and you are mostly stuck with cassandra-stress. (https://www.scylladb.com/test-drive/) Better than nothing, but not perfect.
For your testing, we are happy to share all the necessary instructions on how to get started using Scylla with Kubernetes and give you access to our Helm Charts and stateful sets. We are also happy to make ourselves available over Slack and/or web conferences to help set everything up.
Hardware requirements are https://docs.scylladb.com/getting-started/system-requirements/. Scylla will probably run on your laptop, but the official testing and Production recommendations are:
Installation
Cores
Memory
Disk
Network
Test, minimal
4
2 GB
Single plain SSD
1 Gbps
Production
20 cores - 2 socket, 10 cores each
128 GB
RAID-0, 4 SSDs, 1-5TBs
10 Gbps
Ultimately, if you can share workload specs with us, we can make hardware sizing recommendations.
Slack invitations are available at http://slack.scylladb.com. Binaries are at https://www.scylladb.com/download/#binary (or https://www.scylladb.com/download/#docker). Let us know when you have the cycles to do some testing.
Greg
On Mon, Dec 10, 2018 at 4:35 PM Keong Lim <Keong.Lim@huawei.com> wrote:
Hi Greg,
To that end, do you have some public demo lab available that runs Scylla in a Kubernetes cluster, where we could push some data and queries through to observe the behavior and collect stats?
What would be the min/max/default CPU/RAM resources to configure in the pods?
ONAP project has its own Cassandra clusters running in various test labs, so a virtual-side-by-side comparison might provide an interesting data point to consider.
Keong
From: Greg Matza [mailto:greg@scylladb.com]
Sent: Tuesday, 11 December 2018 11:02
To: Keong Lim <Keong.Lim@huawei.com>
Cc: stephen.terrill@ericsson.com; jf2512@att.com
Subject: Re: ScyllaDB
Sounds good.
Of course, Jackson vulnerabilities may be the bleeding wound that needs to be patched up most urgently. But there are also other benefits of Scylla vs. Cassandra. Namely, easier maintenance (no JVM tuning, no cache tuning, etc.), and typically cheaper/better performance on fewer nodes.
I just mention this because the idea of Scylla as a wholesale replacement for all the C* throughout AAI (and even ONAP) came up on your last discussion. (I listened to the recording) Obviously, we'd encourage that! We think our software is awesome! But, beyond our boosterism, there might be real value to users, beyond the Security aspects.
Look forward to hearing from you,
Greg
On Mon, Dec 10, 2018 at 3:56 PM Keong Lim <Keong.Lim@huawei.com> wrote:
Hi Greg,
The issue of replacing Jackson libraries came up again from a security standpoint, so there may be some appetite to look at a Cassandra replacement as part of that work.
The discussion right now is about which use cases will be included in the next release, but priorities have not been agreed yet.
Will keep Scylla in mind when the topic does come up again!
Keong
From: Greg Matza [mailto:greg@scylladb.com]
Sent: Tuesday, 11 December 2018 10:28
To: Keong Lim <Keong.Lim@huawei.com>
Subject: Re: ScyllaDB
Keong,
Congrats on the Casablanca release.
We're here to assist with any testing or discussion, once the Cassandra/Scylla question comes up again, as part of Dublin. Any thoughts on when that might be?
Greg
Quick CVE comparison:
Actions:
For AAI project: