Getting Started with Alpine Linux and Cloudsmith

How to get set up and started with your own private, secure repository for Alpine Linux Packages on Cloudsmith.

https://help.cloudsmith.io/docs/alpine-repository

 preparations are already underway globally. If you're not familiar with the event, it's a month-long celebration of open source during October by Digital Ocean. Hacktoberfest promotes contributions to a diverse range of open source projects on Github and welcomes for first-time contributors to get involved with the community.

It's easy and the perfect time to get involved or even to rejoin the open source community. If contributing to your favorite open source project sounds appealing, wait, because it gets even better! Hacktoberfest is open to everyone across the world. Contributors can sign up anytime between October 1st and October 31st to become edible for fantastic swag in the form of t-shirts and stickers from Digital Ocean. Submitting four quality pull requests to a public Github repository qualifies each contributor to swag, and what fantastic swag it is! Check out last year's fantastic t-shirt!

At Cloudsmith, we use open source tools and dependencies in all of the software we build and use daily. We love giving back to the community and frequently contributing enhancements and bug fixes to libraries and package management tools to help our customers and our own internal needs.

Helping to support the open source community is important to us. That's why we provide 

 for all our package formats and provide open source tools to help users get the most out of Cloudsmith as their universal package repository.

This year, like many previous years, will see members of the team contributing to a range of open source projects for fun and to scratch our own itch by making something we love even better. Still, we're all about giving back and want to help empower developers to contribute to tools they enjoy using. That's why we will be creating a range of issues suitable for first-time contributors and experienced developers to make code and documentation improvements across some of our open source projects:

As a team, we are looking forward to engaging with the community we love, and creating new awesome software together!

A month-long celebration to open source projects: coming soon!

At Cloudsmith, we pride ourselves in being the central source of truth for your packages and build artifacts. We provide one global platform to centralize your assets and give you the control and visibility that you need as part of a modern DevOps process.

As part of that effort, we’re delighted to announce that if you’re using AWS CodeBuild as part of your CI/CD workflow and pipelines, we can offer you a simple way to get the output of your build processes into your Cloudsmith repository

AWS CodeBuild is a fully managed continuous integration service that allows you to automate your source compilation, test running and packaging of your software. There is nothing to install or maintain, and you can be up and running in minutes with your own build pipeline.

Why would I want to integrate Cloudsmith with CodeBuild?

Much like CodeBuild, Cloudsmith is a fully managed repository service. And just like CodeBuild, there is nothing to install or maintain, we take care of all of that for you.

When you automate your build process with CodeBuild, you’ll need somewhere to store the resulting build artifact(s). Of course, you could store these somewhere basic, like an S3 bucket or maybe even an on-premise storage system of some kind. But doing that means you’d miss out on some of the distinct advantages Cloudsmith provides, namely:

 Cloudsmith will deliver your artifacts and packages to customers or distributed teams at the lowest latency possible. Configurable global storage regions, configurable edge caching rules and a fast Global CDN mean users never have to wait to consume or use the output of your builds - no matter where they are.

We provide fine-grained access controls, enabling you to grant teams and end users the precise level of access that they need, and no more - including SAML/SSO integration so that you can plug into your existing enterprise auth systems. We also allow you to create read-only access tokens with additional configurable restrictions - if you are vendoring / selling software to end users, for example.

We support a wide variety of package formats, including raw files, binaries or disk images. So whatever the output of your CodeBuild process is, we can support it - including integration with the native package management tools for most formats. The goal with Cloudsmith is simple: that it just works.

This list is far from exhaustive - there are many, many reasons you’d want to use Cloudsmith as a platform to host and store your build artifact(s). We are happy to talk about this at great length, far longer than we can fit in here 😊 Just contact us if you’d like to chat more about it!

The bottom line: in much the same way you probably wouldn’t entertain the overheads involved in standing up your own server farm to run your CI/CD process, you don’t need to do that either for your package repositories.

Using AWS CodeBuild with Cloudsmith - A worked example.

OK, so how do we go about integrating our Cloudsmith repository with our CodeBuild projects? Let's work through an example.

This example will be pretty straightforward - we will build a 

 package from source stored in a GitHub repository and then push the resulting build to our private Cloudsmith repository.

You use the AWS Console to set up a new CodeBuild Project. Just select “Create Build Project” to get started:

Next, you give your project a name, an optional description and any additional optional tags:

You then select a source for your project, in this case it’s a GitHub repository, but other source repository types are supported:

As shown above, I have configured a GitHub personal access token to allow CodeBuild to access my private GitHub repository.

Then we configure the webhook that we want to use to kick start this build. In this example, we will just start the build on any push to the specified GitHub repository:

Now we select the container environment that we need to run this build, and in this case we will use the latest available Ubuntu image, and we will create a new AWS IAM service role at the same time:

And finally, we will specify that this project is going to use a buildspec file:

A buildspec file is where we define the phases of the build and what commands / actions we wish to perform as part of the build process. We will go into this in more detail in the next step!

You then just click “Create Build Project” to finish.

Our buildspec file contains all the information required to define our build process. We store this buildspec file in our GitHub repository along with the source code that we are building. A buildspec file can be as complex or as simple as your individual project requires, but for this example it’s pretty simple.

You’ll need to set up any environment variables that your build process requires. In our example, we only need a single environment variable; our Cloudsmith API Key. We need this to use the Cloudsmith CLI to push our build artifact to our Cloudsmith repository, and we don’t want to store our API-Key as plain text in the buildspec file as it would then be revealed in any build logs - not good!

To achieve this, we use AWS Secrets Manager to securely store our API-Key, and then we can reference it in our buildspec file as follows:

At a minimum you then, you’ll need 3 phases in your build:

The install phase is where we install any prerequisites that our build needs, and this of course will vary depending on what you are building and what your individual requirements are. In our case that means the required tooling to build our Debian package, and of course the correct Python runtime for the Cloudsmith CLI and the Cloudsmith CLI itself:

The build phase is where we execute the commands required to actually build our package. For our 

This is where the magic happens! In the post build phase, we only require a single command - `cloudsmith push` to upload the result of the build to our Cloudsmith repository

That’s it, that’s all we need to build this package, and store it in our Cloudsmith repository. It really is that simple. Now of course, your build process itself may be a lot more complex, but to add Cloudsmith to your build process really is just two commands:

2. cloudsmith push FORMAT OWNER/REPOSITORY PACKAGE_FILENAME

We don’t get in the way, you don’t need to totally reconfigure your entire build and we try to make it as easy as possible for you to get up and running!

As we configured this CodeBuild project to start on a push to our GitHub repository, If we just make a change to our source and then commit the change the build will start:

The job has been Submitted, Queued and CloudBuild is now provisioning the container to run the build. Once provisioned, we can tail the logs to watch the progress of the build:

When the build finishes, our package has been built successfully and uploaded to our Cloudsmith Repository. If we now login to Cloudsmith and view the repository, we can see our package:

We hope that this demonstrates how simple it is to integrate Cloudsmith with your existing (or new) AWS CodeBuild projects, and gives you an idea of what you can achieve with your own CI/CD pipelines.

If you’d like to try it yourself, you can sign up for a fully-featured free trial at cloudsmith.com, and we are here to help you every step of the way.

We wish you very happy and successful builds!

Integrating AWS CodeBuild with Cloudsmith Repositories

Using Ansible to deploy packages from Cloudsmith

A quick example of building an Ansible playbook to deploy a package from a private Cloudsmith repository.

Using Buildkite to push packages to Cloudsmith

Dan McKinney, DevRel at Cloudsmith, takes a look at how to set up a Buildkite pipeline to push packages to Cloudsmith.

One reason for building a ‘single source of truth’ for 

 is that it gives the organization control over who can use what when.

The ‘wild West’ of public repositories gives no control at all and can lead to a situation in which packages and dependencies of dubious provenance are integrated into builds without a second thought. Within the Cloudsmith world, we want to have the maximum security and control possible.

Permissions can also help the individuals and teams in the organization find what they need faster, by ensuring that the repositories they see are optimized for them alone.

To help make that happen, Cloudsmith includes the ability to create teams and organizations and limit access to repositories at team and individual level. Let’s take a look in more detail about how to make that happen, and discuss other considerations when building out teams and orgs.

It all starts with the organization. Organizations give you the ability to configure access for teams, individuals and machines that map to your companies organizational structure - and building security and resilience into the management of teams and workflows is essential in today's ecosystem.

We recommend creating only one organization in Cloudsmith. You are not limited to one, but as we allow as many teams as you like beneath any organization, there isn’t really a need to create more. Similarly, as all teams and users are associated with a specific organization, more than one of the latter will usually mean replication of effort.

As a further consideration, a single organization helps in terms of transparency as reporting, auditing etc is also delivered at the organization level.

full details are in our documentation here

). Once that is done, you can configure name, billing and contact details and so on before considering global permissions.

At the organization level Cloudsmith allows users to define global privileges for members around inviting new collaborators and members, and creating teams and repositories. In the same way object privileges can be set - that determine the access level members have in terms of access and actions for repositories within that organization.

Similarly, at an organization level you can define usage limits for repositories and check what the current situation is around usage at any time - handy for managing or avoiding unpredictable usage bills.

In order to manage permissions and access at a more granular level, you will want to create teams within organizations. Cloudsmith then enables permissions for teams to be managed at the repository level.

When creating teams, think clearly about 

 between groups within your organization when it comes to package management. You may want to limit publishing packages to repositories (write access) to specific teams, which limiting others to read access, for example.

To put that another way, when giving any team access to a repo, you will be asked to specify the level of access for that team (as shown in the image below), so it makes sense to build out teams with that in mind.

Although teams can help manage permissions quickly and transparently, it is of course also possible to give named individual users access to individual repositories, and specify what level of access they have - in this way individuals not in teams can be given access, and those already in teams can be given improved access over that offered to the team as a whole.

ultimately permissions are controlled at the repository level

. When creating a repository (or indeed at any time after) Cloudsmith enables you to specify precisely what ‘Admin’, ‘Write’ and ‘Read’ access will enable each user or team to perform, and this is specified at the repository level as shown in Figure 2.

This effectively means that a single team (or individual) can have ‘Write’ access to separate repositories that means something slightly different in each case. Ideal for fine-grained control. Cloudsmith is all about securing your software assets. Our platform gives the control to do that. Remember - err on the side of caution!

If you haven't already, try out these features through a free trial of Cloudsmith today. Get started with your first repository within 60 seconds!

Creating Organizations and Teams and Managing Permissions In Cloudsmith

What software assets does your organization use?

What sounds like a simple question is anything but. If we include every package and dependency that ends up in the code we produce then for most development teams the truthful answer is ‘we don’t know’.

As we’ve said enough times already, that really isn’t good enough anymore. And that’s one of the core motivations behind Cloudsmith. A central private repository provides an ‘isolation layer’ in which everything you use is both stored and controlled.

Cloudsmith is entirely flexible in terms of how repositories are built and structured. Depending on what is convenient for your organization, you can either build universal repositories in which packages in multiple formats sit side-by-side, or format specific repositories.

But our support for transparency around the use of software assets across the entire organization doesn’t stop there.

By providing a single repository for all your assets, Cloudsmith enables you to check at a glance exactly which individuals and processes are downloading those assets. This can be helpful in a number of ways. It certainly is if you need to quickly establish precisely where any given package may have been used in the wake of identifying a specific vulnerability.

Download logs in Cloudsmith, as shown below, mean you have a record of precisely what is being used where, and by who or what. At any moment you have a full audit trail and detailed information relating to every download of every asset, so you can ensure the right people are accessing the right things in the right way.

We also provide event logs along similar lines, so you can keep a handle of any actions against any package within your Cloudsmith environment.

Beyond granular level access logs, Cloudsmith also supports aggregated reporting around access to each individual repository. This includes download and upload counts and associated bandwidth, plus downloads and uploads reported against geolocation, package type and distribution.

This type of reporting has a variety of uses. For one, it helps manage your account with Cloudsmith and ensure total visibility around usage, which in turn can be used to accurately forecast costs. (Cloudsmith also supports the configuration of retention/lifetime rules, meaning you can automatically delete or move packages after a certain amount of time in order to actively control those costs).

Full statistical reporting also provides key insight around which packages and formats are most popular, plus which are trending up or down in terms of usage. This data in turn can help understand where the business is particularly exposed to one package or package type, or alternatively which packages are being stored and rarely used.

All in, we provide a variety of methods for understanding how your repositories, packages and assets are used. All of them support better, more efficient development and distribution.