Using CircleCI to push packages to Cloudsmith

A short demo of how you can use a CircleCI pipeline and the Cloudsmith CircleCI Orb to push a package to a private Cloudsmith repository.

Modernizing the Management of Your Software Supply Chain - Tom Gibson - ASW 169

Tom Gibson from Cloudsmith joins SecurityWeekly to discuss the importance of having an SBOM for your organization.

Finding and fixing known vulnerabilities in dependencies and container images

Building a source of truth for packages to avoid malicious packages getting through Combining continuous packaging and security into a CI/CD pipeline

Establishing Trust & Provenance in your Software Supply Chain

Visibility in your Software Supply Chain with upstreams and signatures

Modernizing the Management of Your Software Supply Chain

We are excited to announce that Cloudsmith has secured $15 million of funding in our recent Series A round. This latest round will help us continue to build best-in-class technology for today’s software engineers and their organizations by evolving cloud-native package management and providing a secure, single source of truth for all software artifacts and assets.

Cloudsmith was built to tackle the complexity of managing software assets, a key challenge for any company to secure their supply chains. Especially in 2021.

With the Cloudsmith platform, we enable our customers across the globe to track and control the distribution of any software asset and provide the tooling to minimize the risks of utilizing open-source software.

It’s an industry-wide issue and our investors, led by Tiger Global, understand the need for a fully managed cloud-native solution that Cloudsmith provides.

Shaping the Future of Software Supply Chains

CI/CD and DevOps are part of modern software supply chains, but a genuinely holistic view from source to delivery is required. A solution that powers global infrastructure at scale and provides criticality of performance, observability, and isolation. A logistics-based smart CDN that offers controls and insights; a software-aware Package Distribution Network (PDN).

As engineers ourselves, we understand that ownership of how our customers’ software artifacts, containers, and packages are pulled in, stored, and distributed is vital. It's the central pinch-point that everything should pass through, and going forwards, as supply chain attacks increase, every single opening in your supply chain is a vector for attack.

Our complement of continuous packaging, which is the glue between CI and CD in the supply chain, positions packaged assets as first-class traceable entities in the supply chain's bill of materials.

With support for 24+ formats, 225 PoPs, 24/7 engineer-driven support, and an ever-evolving partner ecosystem with leading tools like Buildkite, HashiCorp, Snyk, and CircleCI, our mission is to evolve the cloud-native supply chain, making it simple for organizations to secure their software delivery at scale.

Venture-Backed by Top Software Automation and DevOps Firms

Our Series A round is led by Tiger Global, with participation from Shasta, Sorenson, Amaranthine, Leadout Capital, and previous investors Frontline, MMC, and Techstart.

Scott Johnston (CEO of Docker), Sarah Friar (CEO of Nextdoor), Yvonne Wassenaar (CEO of Puppet), Shane Curran (CEO of Evervault), Matt Klein (Lyft, creator of Envoy), Yun-Fang Juan (ex-Facebook), Cian O'Maidin (CEO of NearForm), Babak Hamadani (Primeset, ex-Twitter) and Kit Merker (Nobl9) also participated in the round, raising our total capital to $15 million.

We are also incredibly lucky to have these awesome Angel investors who participated in the round that truly understand and believe in the Cloudsmith mission.

Our new funds will be used to grow the Cloudsmith team, deliver on our product roadmap, and continue to expand into international markets.

To every Cloudsmither, partner, customer, and advisor of Cloudsmith, thank you for your continued support. We honestly couldn’t have done this without each and every one of you.

Thank you for choosing us and believing in our product and our mission. We may be five years in, but this is only just the beginning.

We’re beyond excited for this next phase of Cloudsmith, and we’re currently hiring across all departments.

 and come help us revolutionize cloud-native supply chains!

Cloudsmith Raises $15m in Series A to Evolve the Future of Software Supply Chains

Package repositories were never something I thought about as a developer unless something didn’t work. For example, if it was slow, wouldn’t connect, wouldn’t install, or was overly complicated to configure. Mostly I wanted something I barely noticed. Something simple and easy to use.

When I started my career, I was developing Windows apps in the late 2000s. My packages were the exes and dlls produced by my lovely big Visual Studio IDE. Our dependencies were either official windows libraries or proprietary 3rd party licensed libraries. The exes and dlls were stored in folders on an FTP server and could be accessed using Microsoft access controls- so I suppose the FTP server was my package repository. The software ran on on-premise hardware, and the users were an in-house team that I knew well. I owned the application and the entire software workflow.

That software development workflow is very different from a more modern workflow where:

I work on a small part of a bigger project which uses multiple frameworks, languages, and containers.

My code is dependent on open source software (OSS) hosted externally.

Multiple developers work on the same codebase simultaneously and make frequent commits to the code repository.

Code could be built, scanned, and deployed many times a day via a CI/CD pipeline

The output of the build (the package) is deployed to the cloud

Distribution of the software package to customers is tracked and controlled

As a developer, I want my package repository:

To have one place to view and control all my software packages in all formats- a single source of truth

To push/pull packages super fast for all regions my teams are based

To handle distributing my software to customers.

To be able to sign my packages and have that managed for me

To integrate easily with my other build and security tools in an automated way.

Something simple and easy to use with extensive documents

This was meant to be one blog- but it turns out package management is a dark horse and needed two blogs!

This blog will give a few explanations that we in the package management industry throw around. Our next article on the topic: 

Private Package Repositories Part 2: The Influencers

 will go into the things that have influenced package management.

A software package, artifact, or image is the output of building software- it groups together files containing your software along with the metadata about the software and dependencies in a well-defined format. Packages are typically versioned to provide a better and more manageable understanding of what software is being deployed.

Packages promote the reuse of code as it can be dropped into another application and used easily. Packages are created using a package manager and are usually stored in a repository, like Cloudsmith. The table below details some common packages.

Package metadata describes the package with information about the author, repository location, repository version, file type, license, package dependencies, and more. Metadata can also have information about the CI/CD build like who triggered the build, the build time, approval information, vulnerability information, or user-created metadata.

A package manager is software that creates, uploads, installs, upgrades, and configures software packages for a language, container, or OS. Every package type gets its own unique package manager: Debian’s RPMs use apt-get, Node’s packages use NPM, Python’s packages use Pip, etc.

Some packages have more than one Package Manager to choose from. For example, .NET’s NuGet packages can use 

 or the native NuGet package manager. Similarly, Java’s Maven packages can use the native Maven package manager, Gradle or Ivy. Some package managers are more stable, easier to use, have faster build times, or have access to different packages available in their repositories.

A package repository, registry, or feed is a place to store all of your packages.

Package repositories work closely with package managers, and the terms can get mangled when talking about software tools, like Cloudsmith or JFrog’s Artifactory, that support most software packages. The terms become mingled because these tools have the functionality of a package manager to upload, download and configure packages, and they also host all the packages on repositories.

It is quite a task to provide support for many formats as every language/OS has its unique package manager.

If a package uses another package, that package is called a dependency. Almost every project uses third-party packages as libraries and/or frameworks.

Resolving dependencies in a package is no joke- specifying and resolving the dependencies and relationships between libraries and packages is one of those 

problems. Version constraints on packages mean package managers have to solve a problem equivalent to SAT solving.

On top of dependency resolution being a complex problem to solve, different packages have different ways to resolve conflicts or missing dependencies and some package types have deep dependency trees (I’m looking at you NPM). All in all, dependency solving is a toughie.

 allow you to store packages of different types in one repository. Many package managers don’t let you store different packages in the same repository. Multi-format repositories are especially useful if your tech stack uses multiple languages and containers - read more in our article on 

why modern tech stacks need multi-format repositories

.

Multiformat Repositories means fewer repositories to manage- I think that is a good thing.

Many languages and containers provide a Public repository to host your packages. NPM, for example, provides npm public registry, and Python provides the PyPI repository.

Publicly available packages have made it so much easier to use Open Source Software (OSS) and have changed how software is built and deployed forever.

The benefits of OSS for organizations are numerous. Still, community-controlled public repositories cannot guarantee availability, bring an increased risk of introducing security vulnerabilities, generally only host one package format, and cannot control who downloads your package.

Many organizations need a private repository for their packages for 

, compliance, availability, or reliability. On top of that, private repositories provide additional features required by enterprises, such as:

Access Controls for Teams and Entitlement tokens

Integration with CI/CD and Security tools

Service Level Agreements guaranteeing uptimes

Package Upstreams allow users to consume packages hosted elsewhere from public repositories like Maven Central, PyPI, NuGet.org, npmjs.com, or Debian’s package registry. When a repository has an upstream configured, the service regularly checks for new upstream packages and stores them in the private repository

The rules around the order and precedence of what repositories to search and what packages to select will determine what packages are used. Generally, packages in the repository itself supersede packages from the upstream.

Upstreams allow dependencies to be isolated from untrusted 3rd party sources to protect you from outages and slowness of external services.

The whole point of signing a package is to be able to 

 that a package is safe to download or use as a dependency.

Many software organizations use their own GPG/RSA key for signing their metadata and packages which are usually managed by the private repository. Signing a package with your organization’s key lets Developers know that this package was written and approved by your organization.

Lately, the software community is coming to grips with how OSS software can be the source of entry in supply chain attacks. It’s a lot harder to trust code that was not written and signed by your organization. Signing OSS packages can help but even if an OSS package is signed it is not clear if you should trust it.

In the absence of using a trusted signed OSS package, package repositories can scan OSS packages for known vulnerabilities and extract metadata information like version, who wrote the code, results from scans, or license information which can provide insight into the provenance of the software package.

 has been working to improve trust in OSS software by improving transparency and simplifying the signing process by providing a service for package signing similar to 

Private package repositories need to be able to manage signing packages and maintain keys, work with a package’s native signing tools, and work towards integrating with new tools to sign and trust OSS packages.

The Software Bill of Materials (SBOM) is essentially a list of all components, including licenses and dependencies contained in a software product. Most software includes dependencies sourced from the open-source community or commercial software. New regulations have come in to 

publish the SBOM of software used by US government departments

The end-user of software can use the SBOM to perform vulnerability and license analysis of their software packages which can evaluate risk in a software product.

The Software Package Data Exchange (SPDX) is an open standard for communicating the SBOM. SPDX has become 

 as the standard for communicating the SBOM. SPDX is integral to generating an SBOM that can be easily shared and automated.

Explore the current state of software supply chains, and why Continuous Packaging is critical to secure CI/CD pipelines in our newly released report with O'Reilly.

A software license agreement is a legal document chosen by a software company or developer on how a user can use the software and should be included in the package. There are many software licenses with different legal terms, support agreements, limitations, and costs.

Most software licences fall into two groups either:

Free and Open Source Software (FOSS), e.g., GNU General Public Licence (GPL), Apache, BSD, and MIT

Proprietary software License, e.g., EULA.

A package manager should match the license defined within a package's metadata as accurately as possible. For example, the BSD license specified within this package's metadata is checked against a valid SPDX license. Adding this license reporting functionality gives Developers more visibility, control, and management across all aspects of your package management.

Software vendors can use private package repositories to distribute their software. Private repositories that can distribute software packages eliminate the need to rehost the package elsewhere for customers and all the management associated with that.

Software vendors distributing software may need their private repository to:

Provide a reliable and fast software package distribution.

Control who downloads your package. Cloudsmith does this with 

Provide Service Level Agreements (SLAs) to guarantee service levels.

A content delivery network (CDN) refers to a geographically distributed group of servers that work together to provide fast delivery of Internet content.

At Cloudsmith, we developed what we call the Package Delivery Network, or PDN. It’s like a highly customized CDN that knows that it deals with packages, package authentication, and client package management tooling. It helps deliver packages faster to distributed users.

Package management was a lot simpler at the start of my tech career when it was just me, an exe, and an FTP server!

Package management has always been complex, even when dealing with a more straightforward landscape of a small number of dependent packages dealing with only one package format. Modern package repositories need to host many formats, deal with complicated dependencies from many feeds while dealing with the problems of scaling, distribution, and security.

Check out part 2 of this blog below, where we delve into trends in the software landscape that have changed what developers and organizations want from a package repository.

Private Package Repositories Part 2: The Influencers | CloudsmithIn this part 2 of the package repository series, we will dive into trends within the software landscape that have changed what developers and organizations want from a package repository.CloudsmithCiara Carey

Private Package Repositories Part 1: What’s a package again?

LF Live Webinar: Continuous...Everything? Integration, Packaging, Delivery, and Security All in One!

Application delivery pipelines share many traits with any assembly line.

Open source libraries, runtimes, databases, and base images are raw materials to be inspected to ensure they meet quality and security standards. Modern applications combine these raw materials with custom code to create dynamic and complex environments, opening up an extremely effective attack vector.

While not new, software supply chain attacks are on the rise with changes in how applications are built. With open source components making up more than 80% of modern codebases, having a reliable “source of truth” for the security, compliance, and provenance of software and its dependencies is critical.

In this webinar, Snyk, CircleCI and Cloudsmith will share practices and tools that are helping developers create applications with integrity, quality, and security.

Find and fix known vulnerabilities in app dependencies and container images

Build a source of truth for open source packages to avoid malicious packages

Combine continuous packaging and security into a CircleCI delivery pipeline Create provenance and security quality gates as part of deployment workflows

Join Ryan Pedersen (CircleCI), Tom Gibson (Cloudsmith), and Tomas Gonzalez (Snyk) to learn how to manage and mitigate this new risk profile and create a more holistic application security approach.

Continuous...Everything? Integration, Packaging, Delivery, and Security All in One!

Deep in the woods, where trees are black and the air is thick, steam rises wistfully across the damp ground. A single dirt track, barely wide enough to pass, scars the terrain for what seems like an endless number of miles. It winds its way through the mountains and valleys, across a rickety bridge over a cavernous ravine, before plunging back into darkness, the trees bending over as if to grasp those passing through. Finally, in a small clearing, a lonely decrepit wooden cabin reveals itself.

The cabin sits unobtrusively in its setting, a small ranch porch makes it almost look inviting. The surrounding trees sway and creak in the wind as if whispering a foreboding message of warning. Evil lurks all around it.

But it is deep within that the problem lies.

In the rotting floor, a trap door reveals a rickety staircase which leads ominously down into the blackness below. A single candle is enough to illuminate a narrow claustrophobic path, deep into the tight bowels of the cellar. The stale air is suffocating. Down the passage, through an opening in the rock-built foundations, a small room houses an old oak desk sitting dusty and alone. The world has forgotten it. In the bottom drawer, locked with a key long lost, is a book wrapped in a thick, almost rigid cloth. Within this book are the incantations to unlock a devastating demonic possession.

A public software repository is the cabin. The book is the dependency you're not tracking. The aimless, unwitting temporary residents are the developers and processes used to build your software supply chain. Without knowledge. Without protection. Bad things will happen.

And without realizing it, the incantations are spoken, and the demonic possession is unleashed. You’ve opened a door, a conduit to another place. A scary and unrelenting place. Filled with demons that want your soul. And a vessel to infect the rest of your world. A host to do terrible, horrible things. Things that no one wants or expects. Control is lost.

So fire up the chainsaw and load your shotgun… tackle the demons head on.

Or better yet. Be smart and just don’t open the Necronomicon.

Dead Evil: A Software Supply Chain Possession

Cloudsmith accepted to The Northern Ireland Cyber Security Cluster - companies developing world-leading cybersecurity technologies.

We were delighted to have joined The Northern Ireland Security Cluster - NICYBER. The cluster consists of companies developing world-leading cybersecurity technologies from Northern Ireland, supported by Queen’s University Belfast Centre for Secure Information Technologies 

, the UK’s leading university cybersecurity research centre.

Security is a many-headed Hydra. As soon as you've addressed one aspect of securing your software or infrastructure, more attack vectors appear. You can never be secure enough.

That's why at Cloudsmith we're pushing the concept of Universal Provenance as a key aspect of your company's security solution.

We see Universal Provenance as integral to the shift-left initiative - moving quality and security considerations towards the developer - by using packaging to create logical units of software that can be versioned and tested early in the development lifecycle.

Cloudsmith provides tooling for making the storing, sharing and controlling of software assets easy. This enables developers and teams to secure their IP in a centralised, highly available and traceable repository - all while reducing costs and complexity, and accelerating development cycles.

Security is inherent to DevOps best practices. All modern software pipelines require detailed audit trails to quickly identify issues, weed out bad actors and analyse potential vulnerabilities.

 service gives organisations the high-fidelity control plane they need to understand the provenance of their deployments - which is vital for minimising or eliminating modern security risks.

Using CircleCI to Push Packages to Cloudsmith

More articles

Modernizing the Management of Your Software Supply Chain

Cloudsmith Raises $15m in Series A to Evolve the Future of Software Supply Chains

Private Package Repositories Part 1: What’s a package again?

Continuous...Everything? Integration, Packaging, Delivery, and Security All in One!

Dead Evil: A Software Supply Chain Possession

Cloudsmith joins NICyber Security Cluster