Observability is emerging as critical DevOps practice to predict, prevent, identify, and address issues in the distributed, dynamic, complex, and constantly evolving applications and infrastructures that characterize modern software systems. Wisely, many teams are adopting observability tools like Datadog, Elastic, and AWS Athena to aggregate logs from their build tooling and better visualize and understand the state of their entire system.

But, observability tools are only as good as the logs they consume.

A high-caliber artifact management solution should be a rich, central source of information about your software supply chain; after all, it hosts your software and your dependencies. This is why we invest heavily in the quality of Cloudsmith’s logs and make sure they’re easy for you to extract your logs and integrate into your observability tools.

Sharpening your vision with Cloudsmith logs

Our logs give you granular visibility into your entire lifecycle of artifacts, user interactions, and system behavior so you can proactively monitor, debug, and optimize your software supply chain and operations. We provide these three main types of logs:

Reduce risk and react faster with audit logs

 audit logs are essential for robust cybersecurity and compliance strategies. Organizational audit logs capture events across your organization, including the creation and deletion of repositories and modifications to settings.

Repository audit logs offer a timeline of non-package actions within a repo (typically administrative activities), like modifying retention rules or creating an entitlement token.

Together, they offer a comprehensive view, and analyzing these detailed records can be extremely useful for the following:

Identify and respond to unauthorized access attempts or suspicious activities in the audit logs.

Trigger alerts for any unusual patterns or anomalies in user activities that indicate a potential security threat.

Track changes in repository settings or retention rules to ensure compliance with organizational policies.

Monitor and audit user actions to meet compliance requirements and regulatory standards.

Investigate and troubleshoot specific issues reported by users or identified through audit logs.

Utilize historical audit logs to trace changes and events leading up to an issue and rectify it quickly.

Identify and resolve issues promptly by reviewing audit logs for any operational discrepancies or errors.

Use audit logs to track repository creation/deletion and help in capacity planning.

Improve OpEx and product planning with client logs

 help you understand how your builds are being consumed by providing a record of every package downloaded by your users, along with information such as date and time, user agent, and IP address.

You can leverage these logs to improve product and operational decisions, for example:

Analyze client logs to understand how packages are being consumed to help you optimize infrastructure and resource allocation.

Monitor package downloads and associated bandwidth usage.

Implement alerts based on download thresholds to proactively manage and optimize cloud costs.

Analyze download trends to align resource provisioning with actual usage, preventing unnecessary expenditure and optimizing your cloud bill.

Use client logs to identify popular packages, enabling prioritization in product development efforts.

Analyze download trends by country and usage by package versions to inform feature improvements.

Anticipate capacity changes with package logs

 provide a detailed record of all package-related events within a repository.

These include unique event IDs, package IDs, event types, dates/times, user IP addresses, and user account details, and they’re useful for:

Analyze package logs to understand resource utilization trends and plan infrastructure scaling accordingly.

 Identify patterns in package downloads, helping you understand peak usage times and plan for increased capacity during high-demand periods.

 Analyze user locations from IP addresses, allowing you to anticipate regional variations in demand and optimize resources accordingly.

 Identify frequently downloaded packages, enabling proactive scaling for high-demand software.

Compatible for convenient extraction and analysis

You can view all of our logs in your Cloudsmith UI, and we provide help documentation for our

. But you can also easily export and integrate them with your observability tools.

access real-time audit logs via the Cloudsmith API for your 

Most observability tools can consume logs from an AWS S3 bucket, either by ingesting data from custom RESTful API or through integrations.

Datadog is a cloud-based monitoring and analytics platform that can 

, providing insights into system performance. The 

, ingests audit logs information, incorporates other status details from Cloudsmith, and displays them on a Datadog dashboard.

Part of the Elastic Stack, Elasticsearch is a distributed search and analytics engine that can 

 consumed from an S3 bucket. Alternatively, 

can be used for collecting custom HTTPJSON events

Splunk is a widely-used platform for searching, monitoring, and analyzing machine-generated data, including logs. You can 

use Splunk to ingest Cloudsmith logs from the AWS S3 bucket

A serverless query service, Amazon Athena allows you to analyze data directly in an Amazon S3 bucket using standard SQL. We have documentation on 

how to use AWS Athena to extract insights from our Client logs

Cloudsmith logs are crucial for enhancing security, compliance, and operational efficiency in your software supply chain. 

 to empower your organization with actionable insights. Or 

 with any questions. We’re more than happy to help.

Improving Observability With Cloudsmith Logs

Are your CI/CD pipelines at risk? They might be if you use long-lived, static credentials and tokens. Long-lived, static credentials and tokens are one of the most common causes of

CI/CD tools need access to cloud services to publish artifacts, deploy software, and access resources on their cloud provider. So, they need credentials. It's tempting to hard-code them.

 left lots of organizations scrambling to rotate and disable API keys they were using to run their builds.

What's the solution? OpenID Connect! OIDC tokens are a more secure way to handle authentication than long-lived credentials, and they remove the need to store your credentials in your CI/CD platform.

 natively. You can use OIDC tokens to authenticate against our API, CLI and our users can even authenticate using OIDC with format-specific endpoints like Ruby, NuGet, Terraform, or Docker.

A classic but flawed workflow using long-lived tokens.

Workflow with Cloudsmith, GitHub Actions using OICD.

CI/CD platforms like CircleCI and Github actions usually require access to resources on external services, like your artifacts hosted on Cloudsmith’s artifact repository.

Hard-coded credentials stored on your CI/CD tool are often used to authenticate into these external services to allow for a fully automated build. Unfortunately, these hard-coded credentials could allow an attacker to authenticate into your system.

A classic but flawed workflow with GitHub Actions and Cloudsmith could flow like the following:

In Cloudsmith you create a user or service account

 named CLOUDSMITH_API_KEY, with the value of your API key.

In GitHub Actions write some code to build a package and use the 

As part of the build, Github Actions publishes to Cloudsmith using the long-lived token in Github secrets to authenticate into Cloudsmith.

The issue with the workflow above is that if this API token is stolen or leaked there is a risk of unauthorized access.

The workflow above can be made more secure by rotating your keys but this is an extra overhead on development, because you have to do it manually, and it might not actually get done. For this reason, Cloudsmith has introduced authentication policies to enforce 

 without having to manually rotate their keys.

Until recently, storing hard-coded credentials in your CI/CD for external cloud services was necessary if you wanted a fully automated build but there is a better way.

 is a better approach to authentication between CI/CD tools than using hard-coded credentials or long-lived API tokens.

OICD extends the OAuth 2.0 protocol for secure single sign-on (SSO) to authenticate API access control. OIDC uses 

 to allow third-party applications to verify identities and obtain basic user profile information.

Using OIDC tokens your CI/CD platform can securely authenticate into Cloudsmith without long-lived credentials being stored in your CI/CD.

By updating your workflows to use OIDC tokens, you can adopt the following good security practices:

No need to store your long-lived secrets in your CI/CD.

You have more granular control over how workflows can use credentials.

No need to rotate your API tokens. With OIDC, your provider issues a short-lived access token that automatically expires.

Security Assertion Markup Language (SAML) is another authentication protocol for SSO that enables secure authentication between different services.

SAML, unlike OIDC, works only on XML and is more difficult to use in use cases involving services authenticating over APIs.

OIDC is not intrinsically more secure than SAML but is easier to implement especially when working with APIs.

Workflow with Cloudsmith, GitHub Actions using OICD

Our updated workflow using Github Actions and Cloudsmith with OIDC looks like the following:

 as a provider for your Organization. Cloudsmith recommends adding claims here to restrict the context of any requests from the provider.

Update your Github Actions from above to use OIDC.

Add a new job to generate an OIDC JWT token in GitHub Actions. The JWT token generated by the OIDC endpoint can be used as an API key.

Save the JWT token to an environment variable called CLOUDSMITH_API_KEY and do not include the API key in the .yaml push action, the API key will be taken from the environment variable instead.

Trigger a build in GitHub Actions and OIDC will be used to authenticate into Cloudsmith.

This token works with all Cloudsmith API endpoints to manage resources, 

as well as format-specific endpoints, e.g. the Ruby endpoint to get all available Ruby packages.

Cloudsmith’s support for OIDC authentication brings a new level of security and convenience to software engineering workflows.

Hard-coded credentials leave you vulnerable in an attack, and manual key rotation can get missed. Give OIDC a try in your Cloudsmith org by consulting our 

If you don’t have a Cloudsmith org, or you want your own development sandbox separate from your company’s production org, 

Cloudsmith blog

Improving Observability With Cloudsmith Logs

Securely Connect Cloudsmith to your CI/CD using OIDC Authentication