Cloud Asset Inventory Blog Posts

Resoto offers an open-source approach to use infrastructure data to solve today's most pressing cloud problems: cost, resilience, security, and compliance.

In this post, I share the story of how D2iQ, an enterprise Kubernetes provider, used Resoto to reduce their annual cloud bill by 78% and save $5.4M per year.

As companies grow, their cloud infrastructure quickly becomes fragmented and gets out of control. Data about what resources exist and how resources they relate to each other is tedious to acquire.

In practice, this means that the infrastructure layer often remains a mystery, and engineering teams are unable to see what's happening in their infrastructure. This makes capacity planning impossible, limits organizations' ability to control cloud costs, and leaves teams in the dark about potential security vulnerabilities.

The data to understand cloud growth exists as cloud resource metadata describing the state, configuration, and dependencies of cloud resources. Acquiring and unifying this "infrastructure data" into a single place is the solution for a lot of the problems that infrastructure engineers deal with today—not just cost, but also security and reliability.

But infrastructure is fragmented. Data is locked behind cloud APIs, and the tools that use those APIs to control the deployment of cloud resources. In this post, I'll explain how Resoto acquires infrastructure data, and then uses that data to write code.

EC2 instances often account for the largest portion of your AWS bill. Yet, it's notoriously difficult to get a simple list of all EC2 instances across all regions and accounts, as threads on StackOverflow and Reddit show.

You also then want to use that list to ask questions about your inventory, such as:

• How many total instances are there?
• Which instances are running?
• Which instances are missing tags?
• Which resources have an expiration date?

In this post, I'll describe how to use Resoto to build an EC2 cloud asset inventory. The baseline inventory is a list with all EC2 instances, which you then can use to create more narrow and detailed views.

Resoto makes it easy to create an inventory of your cloud infrastructure. With just a few commands, you get a snapshot of your resources. And with that data ingested into a time series database, you not only know the current state of your cloud but also have insights into changes and trends.

Resoto also offers simple and human-friendly ways to view this data—you can export the data to Google Sheets to create charts or use Resoto Notebook for interactive search visualization and analysis.

Today, we are introducing a new way to visualize this data: Resoto Dashboards.

With Resoto Dashboards, your searches and queries are executed periodically and the results displayed in widgets automatically. You can also customize and organize widgets in the dashboard to keep the most important metrics always within reach. You can even share dashboards across your organization so others can access the data. And of course, you can access your dashboards anywhere you have access to your Resoto installation.

"A generation which ignores history has no past—and no future."
— Robert A. Heinlein

While Heinlein's words refer to human history, they also apply to cloud infrastructure. Most of the time, we care about the current state of resources; but sometimes, we want to know the origin of a resource, when a resource was deleted, or when/how a resource was updated.

Such knowledge is necessary in situations where you need to understand the timeline to investigate a specific system behaviour:

• To perform the post-mortem analysis of an outage, we need to know which cloud resources changed and how they changed to yield the behaviour that we saw in our application. Without the ability to review a change log this becomes impossible.
• To understand cost spikes in your cloud billing dashboard, you need to understand what resources were created, when they were created, and by whom they were created. Not only do you need a list of changes, but also the ability to filter, group, sort, and aggregate the data to see the big picture.
• To check for security issues or compliance violations, you may need to reduce the scope to verify only those resources that were created or updated since the previous scan. Even complex checks can be performed on large infrastructures if they are only run against changed resources.

History is a log of events defining your infrastructure. This event log is important, as it will enable you to answer future questions about the state of your infrastructure retrospectively, including tomorrow's questions that have not yet crossed your mind.