Who would have thought software could rattle the White House? But a vulnerability in Log4J, a popular open source software project, exposed critical digital infrastructure to remote code execution attacks. This prompted the US Government to engage big tech, infosec professionals, and open source organizations to come together to help secure open source software.
Threats in commonly used open source software (OSS) is the stuff of nightmares. Ransomware attackers have started mass scanning campaigns for vulnerable versions of the Log4j packages. Research from 2020 exposed that a vulnerability called Heartbleed is still exploitable even though a patch was released promptly in 2014. Vulnerabilities have a very long tail - Log4Shell is likely to be a threat for years.
OSS is incredibly positive - without projects like Docker, Kubernetes, Debian, NGINX, Apache, or others, technological innovation would be painfully slow. Its innovation, ease of use, and zero cost meant that nearly every piece of software contains OSS. OSS is everywhere, including data centers, e-commerce, phone networks, mobile devices, and power stations.
OSS is here to stay, and securing open-source supply chains will require a combination of developer-focused tooling, an increase in automation, education, collaboration, and support for OSS maintainers.
In this article, we’ll explore:
- Critical threats in OSS
- The US Government’s response to Log4Shell
- Efforts underway to secure and build trust in OSS
- How you can secure your software pipeline
- Why package management is essential to secure your supply chain
Critical Threats in OSS
Severe vulnerabilities in commonly used OSS that are easy to exploit have wide-reaching effects.
The Cybersecurity and Infrastructure Security Agency Director, Jen Easterly, called Log4Shell “the most serious vulnerability I have seen in my decades-long career.”
Severe vulnerabilities like Log4Shell are not new. Heartbleed was a vulnerability from 2014 that we still feel the ripples of today.
Heartbleed
OpenSSL is an OSS project used by virtually every company in the world. On 7 April 2014, OpenSSL's Heartbleed bug was publicly disclosed as CVE-2014-0160 and fixed. The vulnerability, which had been shipped in OpenSSL, made it possible for hackers to retrieve information such as credit card numbers, cryptography keys, and user credentials right through to private documents and communications from what was meant to be secure transactions. Research by SANS from 2020 exposed that this vulnerability is still exploitable even though a patch was released promptly in 2014.
In the wake of the Heartbleed vulnerability, the Core Infrastructure Initiative (CII) was set up to improve OSS security. The CII has since been replaced by the Open Source Security Foundation (OpenSSF).
Log4Shell
The Log4j library, part of the Apache Software Foundation (ASF), is a general and commonly utilized logging framework used by Java developers worldwide. Log4Shell landed December 2021- a critical vulnerability in Log4j, published as CVE-2021-44228, that could lead to Remote Code Execution- a 10 out of 10 on the risk factor.
Cloudsmith already released a blog on responding to Log4j and released additional functionality to search for dependencies such as log4j across packages stored within Cloudsmith repositories.
Ransomware attackers have started mass scanning campaigns for versions of the Log4j packages vulnerable to Log4Shell. The Log4j vulnerability is likely to be a threat for years.
Developers and organizations can run scripts and scanning tools to assess their exposure to Log4Shell and upgrade to mitigate against this vulnerability. It's another reminder of how critical software supply chain security is and how great the exposure can be when a critical exploit emerges in the wild.
US Government’s response to Log4Shell
Last year, the Whitehouse issued an Executive Order after the fallout of SolarWinds. This kickstarted the use of SBOMs, a flurry of new projects to protect the supply chain, and the rise of OpenSSF. New companies like Chainguard were created, focusing on securing software supply chains.
Early this year, in the wake of the vulnerability exposed in Log4j, The White House convened government and private sector stakeholders to discuss how to improve the security of OSS as it is “a national security concern.” The guest list included representatives from big tech companies and open source organizations - Akamai, Amazon, Apache Software Foundation, Apple, Cloudflare, Meta, GitHub, Google, IBM, Linux Open Source Foundation, Microsoft, Oracle, RedHat, and VMWare.
A few weeks later, the US Senate Homeland Security Committee discussed the Log4j vulnerability and how to prevent open source vulnerabilities in the future. Dr. Trey Herr’s, Director of Cyber Statecraft, testimony about the importance of establishing a model where maintainers are supported to ‘fund the mundane’ was echoed throughout the Senate hearing.
These meetings seemed to have an effect:
- NPM now requires the top 100 maintainers to set up two-factor authentication
- OpenSSF, with support from Microsoft and Google, announced The Alpha Omega Project to improve software supply chain security for OSS projects with an initial investment of $5 million.
In January, the Senate passed two cyber bills - one bill to train federal officials on managing risk in the software supply chain, and another bill to help officials coordinate with other agencies on matters relating to vulnerability disclosure programs, information sharing programs, and improvements to election security infrastructure.
The US Government has effectively used its soft power and is willing to use legislation to help secure OSS.
Efforts to Secure OSS
There has been a massive effort by OSS foundations and the wider OSS community to help maintainers and consumers of OSS build secure software.
Leading many of these initiatives is Open Source Security Foundation (OpenSSF) hosted by the Linux Foundation. The OpenSSF is a cross-industry organization that brings together the industry’s most important open source security initiatives and the individuals and companies that support them. Other foundations that have contributed to improving OSS security include OWASP, npm, Python Foundation, CNCF, government agencies like NTIA, and others.
The collaborative effort and the progress made in the last year have been encouraging. Securing the software supply chain is an area that is still in flux, and the projects that will be used in builds are yet to be decided - the proof of the pudding is in the eating!
Increase trust in builds and artifacts
The whole point of signing a package is to trust that a package is safe to download or use as a dependency. When OSS maintainers sign their packages, it allows people downloading them to know they are official and haven’t been tampered with. Until recently, this process has been costly and required maintainers to manage keys and code-signing certs. Sigstore is a set of tools that developers, software maintainers, package managers, and security experts can use for signing, verifying, and protecting OSS. Its goal is to remove the effort, time, and risk of error that comes with more traditional ways of signing.
Other tools, frameworks, and formats that increase trust in builds and artifacts include:
- In-toto, a framework to secure the integrity of software supply chains
- Gitbom is a tool for building artifact trees that can be used with SBOM formats such as SPDX
- Witness, a tool to prevent tampering of build materials and verifies the integrity of the build process
- The Update Framework, a framework for securing software update systems
- A purl or package URL is an attempt to standardize existing approaches to reliably identify and locate software artifacts.
Provide visibility into the "ingredients" of software through SBOMs
The Software Bill of Materials (SBOM) lists all components, including licenses and dependencies contained in a software product and other data, including checksums. Work has been ongoing into specifying the formats of SBOMs, with SPDX and CycloneDX, and tools to generate and inspect SBOMs. There are several benefits to SBOMs:
- An SBOM in a known format like SPDX or CycloneDX helps drive automation and trigger security alerts.
- An SBOM can be inspected to see if there are any CVEs detected in its dependencies.
- The end-user of software can use the SBOM to perform vulnerability and license analysis of their software packages to evaluate risk in a software product.
- SBOM helps track dependencies more easily and quickly so that vulnerabilities are easier to find and fix.
Harden security in OSS projects:
- OpenSSF announced The Alpha Omega Project in February “to improve software supply chain security for OSS projects through direct engagement of software security experts and automated security testing.” Microsoft and Google have supported this project with an initial investment of $5 million.
- NPM has Enforced 2FA for the top 100 OSS maintainers.
- The Python Software Foundation has been working on initiatives in PyPI to improve the software supply chain security ecosystem, like setting up 2FA, creating a security advisory, and integrating with The Update Framework.
- Work on tracking open source vulnerabilities by GitHub Security Advisories and OSV which is a vulnerability database for open source projects.
- OpenSSF, NTIA, OWASP, CNCF and others have created materials and set up working groups on best practices.
Some other OpenSSF projects include:
- Setting up working groups to identify critical OSS projects to make sure they are supported to maintain updates to a certain baseline level.
- The Scorecards project gives consumers of open-source projects an easy way to judge whether their dependencies are safe.
- Another OpenSSF project, Supply chain Levels for Software Artifacts (SLSA), is a security framework backed by Google to prevent tampering, improve the integrity, and secure packages and infrastructure in your projects, businesses, or enterprises.
How to secure your software pipeline
It's hard even to know what OSS is packaged up in your software, especially when considering transitive dependencies. Developers and Security Engineers need developer-friendly tools to help them increase automation to remove the potential for human error, assess their level of risk by providing visibility into what OSS they depend on, and tools to help them control, mitigate and remediate any risk they do have in their supply chain.
Developers and Organizations can do several things to protect their supply chain:
- Enforcing developers to have multi-factor authentication and strong passwords is an easy win to secure your supply chain. Using tools that accept SAML or passwordless can improve security.
- Maintaining an accurate inventory of third-party software components and open source dependencies, a software bill of materials (SBOM), ideally in a known format like SPDX or Cyclone DX.
- Developers should pin their builds to specific versions to protect against attacks on the supply chain such as Dependency Confusion and Typosqatting attacks.
- Use scanning tools at source, during the build, and at packaging to detect vulnerabilities.
- If you use OSS, bring those packages into your private repository, where you can test/scan against them. Cloudsmith advocates for only exposing development repositories to external public feeds. You can then promote to a production repository with no public feeds after adequately testing and scanning your packages for vulnerabilities.
- Integrate with new tools to sign OSS packages in a meaningful way like Sigstore
- Bring packages into your private repositories to ensure your OSS packages' high availability and protect against outages on public repositories.
Ultimately, developers and their organizations should have a single source of truth for software assets so there is one place to:
- View, search, and audit package metadata
- Manage and audit access
- Streamline automation for your development teams
Package management is central to securing your supply chain
Cloudsmith is a hosted package management service. We securely store your packages in a private artifact repository and to make it easy to automate and integrate your artifact registry with your existing CI/CD pipeline.
Having a single source of truth for your software artifacts is central to securing software supply chains. Cloudsmith’s private multi-format repositories support many package formats and provide one place to track, manage, distribute and understand all software pulled into your stack. A central trusted store forces you to apply processes and controls to that ingress/egress of software packages.
Security is paramount at Cloudsmith as we understand package repositories need robust security features and policies to prove they are trustworthy. That's why:
- Cloudsmith is certified under ISO 272001
- Cloudsmith operates a vulnerability disclosure and Bug Bounty program
- Cloudsmith’s platform has high availability and redundancy. We host Cloudsmith across multiple data centers and multiple regions, with redundancy built-in at each layer of the service and platform.
- Cloudsmith provides robust access control with 2FA for distribution and development.
- Cloudsmith offers support for Single Sign-On (SSO) at the organization level using Security Assertion Markup Language (SAML)
- Cloudsmith provides logs, and Metrics API for observability into what is happening with your packages.
Cloudsmith establishes trust and provenance in your software supply chain, including your OSS 3rd party dependencies, by surfacing your package metadata informing you about the package checksums, how and who built your packages, and what dependencies are in your packages. Cloudsmith also isolates and protects your software supply chain from public upstream sources like Maven Central by proxying or caching your OSS artifacts.
Expanding the ways we help you secure your supply chain
Cloudsmith already helps developers and organizations isolate their software from public OSS feeds, extracts meaningful information from your packages about its dependencies, and allows you to control and visualize all that information in a central location.
Throughout 2022, Cloudsmith will focus on becoming the best solution for organizations looking to secure their software supply chain by focusing on three areas:
- Assessing an organization’s level of risk and providing actionable data to an organization to better secure their supply chain
- Providing tools that allow security teams to manage risk without slowing down their developers. We will enable security teams to block threats from entering their supply chain, and respond quickly when incidents within the industry do occur by having that single pane of glass to understand the impact.
- Integrations with the ecosystem of tools and standards to improve OSS security include sigstore, SLSA, package URL, The Update Framework, in-toto, OSSF Security Scorecards, and more.
At Cloudsmith, we love OSS and have enormous respect for maintainers. Cloudsmith also provides zero-cost repositories for OSS projects. We see ourselves as an essential stakeholder in securing the software supply chain. We are pushing to integrate with emerging tools that make it easy for our developers to know what is in their supply chain, and that makes it easier for developers to be secure by default.
2022 will be a defining year for progress in the security of your 3rd party OSS dependencies.
If you're looking for a simple solution to secure your software artifacts, you can sign up for a free 14-day trial and set up your first repository in just 60 seconds.
Cloudsmith's Shannen Rooney created the artwork for this blog, inspired by the below image:
