Cloud Asset Inventory Blog Posts

"A generation which ignores history has no past—and no future."
— Robert A. Heinlein

While Heinlein's words refer to human history, they also apply to cloud infrastructure. Most of the time, we care about the current state of resources; but sometimes, we want to know the origin of a resource, when a resource was deleted, or when/how a resource was updated.

Such knowledge is necessary in situations where you need to understand the timeline to investigate a specific system behaviour:

• To perform the post-mortem analysis of an outage, we need to know which cloud resources changed and how they changed to yield the behaviour that we saw in our application. Without the ability to review a change log this becomes impossible.
• To understand cost spikes in your cloud billing dashboard, you need to understand what resources were created, when they were created, and by whom they were created. Not only do you need a list of changes, but also the ability to filter, group, sort, and aggregate the data to see the big picture.
• To check for security issues or compliance violations, you may need to reduce the scope to verify only those resources that were created or updated since the previous scan. Even complex checks can be performed on large infrastructures if they are only run against changed resources.

History is a log of events defining your infrastructure. This event log is important, as it will enable you to answer future questions about the state of your infrastructure retrospectively, including tomorrow's questions that have not yet crossed your mind.

Software engineers working with AWS have every cloud service imaginable at their fingertips, and developer velocity could hardly be higher. But, even the most shiny of coins has two sides.

While developers can freely spin up compute instances and databases in addition to less tangible things like Lambda functions or virtual identities—at some point, someone will ask, "What is all of this?"

And as that person hacks away in the CLI trying to get an overview of resources spanning multiple AWS accounts, they will inevitably get frustrated.

While Amazon has been a pioneer in cloud computing and offers the largest array of services, there are some things that just aren't so ideal. Namely, API consistency.

In this post, I describe a few of the challenges and quirks with the AWS API and why we're building Resoto. (Spoiler alert: It is so that you don't have to!)

Today's world of cloud computing is complex. There are many cloud providers, each with their own set of services. Getting insights out of your infrastructure requires specialized understanding of the data from each service.

Cloud Service Diversity​

Properties in different services may have different names but the same meaning, or vice versa. To interpret properties, we need to ensure that values have a defined unit of measurement and one base unit. You can see the challenge if you imagine the many ways you can specify the size of a volume, the number of CPU cores, or even timestamps.

A cloud asset inventory is a complete representation of the resources in your cloud. The job of the inventory is to continuously discover new resources and store data about each individual resource (such as its properties, configurations, and dependencies). Examples of resources not only include compute instances, storage buckets, Kubernetes pods, but also access keys and user and org policies.

In modern cloud-native environments, developers enjoy freedom and permissions to create new resources. The resources in a company's cloud environment can easily number in the hundreds of thousands or millions, resulting in new challenges for infrastructure engineers. One such problem is "infrastructure fragmentation"—resources are distributed across regions, organizations, accounts, and/or projects, and each resource has unique properties and APIs. Coupled with constant change, this fragmentation makes it difficult to keep track of resources, which opens the door to cost problems, security threats, and compliance issues.

A cloud asset inventory solves the infrastructure fragmentation problem by providing complete visibility into all resources from a single place.

Understanding what's running in your cloud infrastructure is important for a number of reasons—for example, security, compliance, and cost.

But sometimes, the cloud feels more like a black box that you're feeding with cash, and in turn it performs the work that makes your business run.

Even those spinning up cloud resources might only be aware of their small slice of the pie. With hundreds of thousands of interconnected resources, it is really hard to know what's going on!

Cloud inventory has become a new type of technical debt, where organizations lose track of their infrastructure and how it relates to the business. Resoto helps to break open the aforementioned black box and eliminate inventory debt.