Getting Started with Docker and Cloudsmith

A short demo of how to get started with Docker and Cloudsmith, including building and pushing a Docker Image to a private Cloudsmith repository and then pulling and running the image.

At Cloudsmith, we are proud of our partnership network. We work hard to collaborate and integrate with the best DevOps tools available. Building performant, modern workflows requires frictionless integrations between tooling.

Integrating Cloudsmith repositories with your CI/CD processes facilitates rapid development. Build, test, and deploy/distribute your software in repeatable, performant ways.

CircleCI is a best-in-class CI/CD platform, providing easy to configure, performant continuous integration/deployment processes. And as Cloudsmith is a CircleCI partner, deploying the results of these processes to your Cloudsmith repositories is simple with our CircleCI Orb.

A CircleCI Orb is a packaged, reusable piece of configuration for CircleCI. It enables you to eliminate repetition in your CircleCI config, helps to automate repeat processes, and helps you integrate CircleCI projects with other tools.

For example, you could install the Cloudsmith CLI as part of your CircleCI config and then use Cloudsmith CLI commands to upload build artifacts to your Cloudsmith repositories. But by using the 

, these additional steps are abstracted away for you, speeding up the development of your CircleCI projects and providing you with a simple, easy-to-use syntax to push your artifacts to Cloudsmith.

The first thing you need to do is create an account on CircleCI. Just like with Cloudsmith, you can get started for free, so you can follow along with the steps in this blog at zero cost.

You create your CircleCI account using your GitHub or Bitbucket credentials, and you then authorize CircleCI to access source repositories on these platforms. Once logged in, on your CircleCI dashboard, you can view a list of your GitHub or Bitbucket projects that you can start building right away:

CircleCI embraces `configuration-as-code`. The entire CI/CD process that runs when triggered by a commit to your source repository is controlled by a single file - the config.yml file. This whole CI/CD process is called a “pipeline.” A pipeline consists of one or more Workflows, Jobs, and Steps.

So let us take a look at how you define a pipeline in a config.yml file.

The first thing that we specify in our pipeline is the CircleCI configuration version, and then immediately following this, we define an orb that we would like to use – the CircleCI Cloudsmith Orb:

Next, we define a Job. Jobs are collections of steps which run any commands or shell scripts that we need. Each job must declare an executor, which is where the job will run. An executor can be a docker container, or a Linux, Windows or macOS VM.

For this example, we declare a Job called “Publish”, which will run on Docker Container with Python 3.7 provided by CircleCI – but you can, of course, use your own Docker images also:

jobs: publish: docker: - image: circleci/python:3.7

Following this, we declare the steps that get executed as part of the Job:

steps: - checkout - cloudsmith/ensure-api-key - cloudsmith/install-cli - run: command: | sudo apt update sudo apt-get install ruby ruby-dev rubygems build-essential -y sudo gem install --no-document fpm make fpm -f -s dir -t deb -v 1.0.1 -n cloudsmith-circleci-demo . - cloudsmith/publish: cloudsmith-repository: demo/examples-repo package-format: deb package-distribution: debian/buster package-path: ./*.deb

The first three steps check out the source from our repository, ensure that we have added our Cloudsmith API key as an 

 in our CircleCI project settings, and install the Cloudsmith CLI.

Next, we run a series of commands that install the dependencies that we need to build our example project. For this example project, we will create a Debian package, and for that, we will use a packaging tool called FPM. FPM is written in Ruby, so we need to install Ruby, some related Ruby packages and FPM itself (as a Ruby gem).

Finally, we declare the cloudsmith/publish step, from our CircleCI Cloudsmith Orb. For this step, we need to declare the Cloudsmith repository that we wish to push our built package to, the distribution and version that the package is for (as it is a Debian package) and the path to the built package itself. And that’s it – that is all the steps needed for this pipeline.

The last thing we add to our pipeline configuration is a workflow. Workflows define a list of jobs and their run order. We only need one workflow for this example pipeline, one that runs the “Publish” job (and therefore, the steps it contains) we declared previously:

workflows: cloudsmith_publish: jobs: - publish

version: 2.1 orbs: cloudsmith: cloudsmith/cloudsmith@1.0.4 jobs: publish: docker: - image: circleci/python:3.7 steps: - checkout - cloudsmith/ensure-api-key - cloudsmith/install-cli - run: command: | sudo apt update sudo apt-get install ruby ruby-dev rubygems build-essential -y sudo gem install --no-document fpm make fpm -f -s dir -t deb -v 1.0.1 -n cloudsmith-circleci-demo . - cloudsmith/publish: cloudsmith-repository: demo/examples-repo package-format: deb package-distribution: debian/buster package-path: ./*.deb workflows: cloudsmith_publish: jobs: - publish

All that is left to do now is add this config.yml file to our source project, in a directory called .circleci at the root of the project, and then commit and push to our Github or Bitbucket repository.

Once we have pushed this (and any subsequent commits) to our source repository, our pipeline will be triggered and will run our workflow:

We can click on the job name (in this case, “publish”) to see detailed console output from each step as it runs:

And once the pipeline has completed successfully, we can check the output from the “Publish Package” step:We can see that our CircleCI pipeline has successfully uploaded our Debian package to our Cloudsmith “examples-repo” repository. If we now login to Cloudsmith and have a look at the repository, we can see the package is present:

Now, I’m sure that the steps involved in building your projects would be much more complicated than the example outlined above. However, thanks to the 

, pushing the result of your build to a Cloudsmith repository should be no more complex than demonstrated.

CircleCI Developer Hub - cloudsmith/cloudsmithInstall the Cloudsmith CLI and publish packages to Cloudsmith.cloudsmith/cloudsmith

The Cloudsmith CircleCI Orb really does simplify the entire process of getting your build artifacts into your own secure, private repository – and then onwards to other CI/CD processes that your build is a dependency for, or even for direct distribution to your customers.

Increase your build velocity, enable repeatable workflows and level up your DevOps, with CircleCI and Cloudsmith. In short, 

Integrating a Cloudsmith repository with a CircleCI pipeline

Like most, 2020 has been a wild year for us here at Cloudsmith - though a great one at that! With new members added to the Cloudsmith family, new customers across the globe, and exciting new partnerships in place, this year was a year of growth (despite us not leaving our houses).

As the year (finally) comes to an end, we're excited to share a look back on everything we've done together and give a massive thank you to everyone who has been involved, whether as a customer, partner, or simply an extension of the Cloudsmith family.

Grab a cup of tea, some biscuits, and enjoy this year's incredible infographic designed by our insanely talented designer, Kimmy.

Cloudsmith By The Numbers in 2020

DevOps "Office Hours" (2020-12-09)

 for this special edition of Office Hours to discuss how Cloudsmith is helping our customers solve the toughest challenges in package management.

Using GitLab to publish packages to Cloudsmith: 

https://www.youtube.com/watch?v=9WpvfegCgBY

Create a Cloudsmith repository in under 60 seconds: 

https://www.youtube.com/watch?v=yzBEWrgbvQQ

https://help.cloudsmith.io/docs/welcome-to-cloudsmith-docs

Edmund, had been promoted to Chief Executive of Elf, Inc. in February, a mere 10 months ago. It had at the time been a great honor, but had almost instantly become a farce. Not only did toy production stop for a number of months, it was now looking like all the toys would remain in storage until such time, Father Christmas would be allowed to deliver them. And who knew when that might be.

The phone rang. Edmund picked up and stoically spoke, “Hallo.”

It was a crazy plan. The elves were to stop all toy production immediately. That was unheard of. Every available toy-making elf had to re-train overnight in software development and related disciplines. That, at least, was possible, elves are extremely adaptable and pick up new information like sponges, and there was a small, but efficient development team that had been building the software for the many electronic toys over the years.

The plan was to build a digital parcel, a package of goodies, for every child who had the means to consume it. Where appropriate it would include video games, e-books, movies, tv boxsets, music, comics, digital trading cards, software, and various digital assets like avatar upgrades and gift cards.

Edmund was horrified at the sheer amount of work involved, this late in the day. He spent hours on a spreadsheet trying to figure out how to achieve this massive undertaking. No matter how often he did the sums, there was no way to build all the packages and the delivery method. They would have to use a third-party to distribute.

He searched for “package distribution” and a few suitable options showed up. After a quick read of each of the offerings feature sets, one stood out, one with enough global infrastructure and scalability to handle such a large distribution all at once.

“This just might work,” he muttered to himself.

Elf Storage and Distribution

The Cloudsmith family is something you have likely seen across our content and communications for years. While many brands make that, well, part of their “branding”, for us, it’s genuinely how we feel about our staff, customers, partners, investors, and everyone we meet along the way. In all cheesiness and truth, we believe the Cloudsmith family is real.

While we’re not based in America and, at the moment, only have one American on staff, Thanksgiving is just as meaningful to us as it is to you. And even though we can’t sit around the table this year to share exactly what we’re thankful for, I wanted to share a message of gratitude on our behalf.

You’re a crazy bunch but we love every single one of you. We love being able to help you succeed with DevOps and pipeline building. And it kills us when we fall short with a missing feature or have to type in those fateful words “It’s on the roadmap.” It is. Thank you for choosing us and believing.

We could not be more excited about building upon our partnerships with CircleCI, HashiCorp, Buildkite, and Cloudposse to name a few. Together we’ve built out some super exciting insights and events for the end of 2020 and early 2021 and for that we thank you. If 2020 is anything to go by, 2021 will be a whole other level of awesome. And for that, we’re grateful for these new relationships we built this year. Watch this space.

And last, but not least, our incredible team…

This holiday season, my thoughts turn gratefully to our employees, advisors, and investors who have made our success possible. Working at Cloudsmith isn’t always easy but I am impressed each and every day with your resilience, resolve, and resourcefulness. Thank you. The donuts are in the mail.

It goes without saying that 2020 has been a wild ride for us all but (a vaccinated) 2021 is going to be EPIC!

Alan

PS: Whether you’re celebrating Thanksgiving or not, here’s a little song to add to your playlist during this very special (yet bizarre) holiday season:

Giving Thanks on Thanksgiving

Universal Package Tags

A walkthrough of universal Package Tags - tag any package or container within Cloudsmith. Use them to search/filter and organize your software assets.