Search Blog Posts

As companies grow, their cloud infrastructure quickly becomes fragmented and gets out of control. Data about what resources exist and how resources they relate to each other is tedious to acquire.

In practice, this means that the infrastructure layer often remains a mystery, and engineering teams are unable to see what's happening in their infrastructure. This makes capacity planning impossible, limits organizations' ability to control cloud costs, and leaves teams in the dark about potential security vulnerabilities.

The data to understand cloud growth exists as cloud resource metadata describing the state, configuration, and dependencies of cloud resources. Acquiring and unifying this "infrastructure data" into a single place is the solution for a lot of the problems that infrastructure engineers deal with today—not just cost, but also security and reliability.

But infrastructure is fragmented. Data is locked behind cloud APIs, and the tools that use those APIs to control the deployment of cloud resources. In this post, I'll explain how Resoto acquires infrastructure data, and then uses that data to write code.

EC2 instances often account for the largest portion of your AWS bill. Yet, it's notoriously difficult to get a simple list of all EC2 instances across all regions and accounts, as threads on StackOverflow and Reddit show.

You also then want to use that list to ask questions about your inventory, such as:

• How many total instances are there?
• Which instances are running?
• Which instances are missing tags?
• Which resources have an expiration date?

In this post, I'll describe how to use Resoto to build an EC2 cloud asset inventory. The baseline inventory is a list with all EC2 instances, which you then can use to create more narrow and detailed views.

Cloud tagging strategies and policies are hailed as one of the most efficient ways to keep your cloud infrastructure controllable. But are they really?

Generally, the idea is that every piece of cloud service gets tagged (or labeled, in case of Google Cloud) by the developers or maintainers who work with it. This could be accomplished with infrastructure-as-code (IaC) tools (such as Terraform), with a command-line interface (CLI), or in the cloud UI.

Cloud Resource Tagging Policies​

Tagging policies could require that each resource needs tags identifying the owner, cost center, product, project, and/or any other metadata. By being diligent about tagging, resources can be managed via their tags and nothing gets overlooked.

Cloud Resource Tagging Challenges​

In theory, this is the correct way to manage resources; in practice, however, this hardly ever works as intended.

Each tag created is a tag that requires maintenance. Tagging policies may change over time and people can make mistakes (in AWS, for example, tag keys are case sensitive).

And, to properly use tagging on a greenfield cloud account is one thing; to retroactively apply tags to sprawling cloud infrastructure is quite another (especially when utilizing a multi-cloud strategy, where you'd need to repeat any operation over multiple interfaces).

In Actionable Cloud Infrastructure Metrics, we explored how to create metrics, export them into a time series database, and visualize them with Grafana. Today, we'll take a look at how to build a web app using Streamlit, a framework that turns data into web apps.

If you are not familiar with Python, don't worry—we're going to keep it simple! In Prerequisites, we'll go over installing Python and the coding techniques utilized in this project.

Today's world of cloud computing is complex. There are many cloud providers, each with their own set of services. Getting insights out of your infrastructure requires specialized understanding of the data from each service.

Cloud Service Diversity​

Properties in different services may have different names but the same meaning, or vice versa. To interpret properties, we need to ensure that values have a defined unit of measurement and one base unit. You can see the challenge if you imagine the many ways you can specify the size of a volume, the number of CPU cores, or even timestamps.