Skip to main content

· 13 min read
Lukas Lösche

Understanding what's running in your cloud infrastructure is important for a number of reasons—for example, security, compliance, and cost.

But sometimes, the cloud feels more like a black box that you're feeding with cash, and in turn it performs the work that makes your business run.

Sheep looking inside a black box

Even those spinning up cloud resources might only be aware of their small slice of the pie. With hundreds of thousands of interconnected resources, it is really hard to know what's going on!

Cloud inventory has become a new type of technical debt, where organizations lose track of their infrastructure and how it relates to the business. Resoto helps to break open the aforementioned black box and eliminate inventory debt.

Resoto provides a searchable snapshot of the current state of your cloud infrastructure, and can automatically react to state changes. Resoto also allows you to aggregate and visualize this data, as my colleagues Matthias and Nikita described in previous blog posts.

Here's an example of a heatmap that allows you to immediately see outliers (like when an account suddenly starts using a large number of expensive, high-core-count instances):

Instance use heatmap

We can ingest this aggregated data into a time series database, such as Prometheus. This information can then be used to build diagrams illustrating cloud resources (e.g., compute instances and storage) over time.

Metrics Overview

This allows you to alert on trends—for example, if you are projected to exceed a quota or spend limit.

Another use case is to quickly identify anomalies using the 3σ rule. If cloud API credentials are leaked or an automated system goes haywire, you would immediately see the spike instead of receiving an unpleasant surprise on your next cloud bill. Best of all, it works across multiple clouds and accounts!

Resoto comes with a handy metrics component, Resoto Metrics, which takes aggregation results and exports them to Prometheus. This blog post describes how to define your own metrics, write some PromQL queries and build a simple metrics dashboard using Resoto Metrics, Prometheus, and Grafana.

· 2 min read
Nikita Melkozerov

We recently released Resoto Notebook, a library that allows for the visualization and exploration of the Resoto graph interactively using Jupyter Notebook.

Resoto Notebook is similar to Resoto Shell in the sense that you execute queries, but the results are returned in a pandas DataFrame structure. This gives you more flexibility in filtering, aggregation, visualization, etc.

info

Please see Resoto Notebook for installation instructions and usage details.

An example notebook is available on GitHub. We encourage you to copy and modify it to suit your needs.

Heatmaps

Resoto Notebook includes the Plotly Python library, which is capable of visualizing data in a variety of ways.

Heatmaps are one of the supported visualization methods. Heatmaps make it easy to spot outliers—imagine, for example, someone was experimenting with an expensive cluster and forgot to shut it down.

Here is an example of a heatmap depicting the number of instances per core, per account:

Heatmap

The y-axis represents the number of cores per instance, while account IDs are listed along x-axis. The color of the heatmap cell indicates the number of instances with the given number of cores; the brighter the color, the greater the number.

Graphs

There are instances where you may be interested in the relationships between resources. Let's say you want to remove a database, but you are not sure of the impact of the removal on other resources. With Resoto Notebook, you can inspect a resource and its relations to see what's going on.

Let's graph a cloud do (DigitalOcean) and two levels of successor resources:

Heatmap

Aggregation

As mentioned previously, Resoto Notebook allows you to harness the power of pandas, a popular Python package for data analysis. The pandas DataFrame structure is a table-like object that allows for easy querying, filtering, and aggregation of data.

Let's try aggregating the number of cores in running instances per account, per region:

Aggregation

With this result, we can quickly identify expensive accounts and act accordingly (e.g., taking cost reduction measures).

Next Steps

I hope this blog post has piqued your interest, and that you will try installing Resoto Notebook. Happy exploring!

· 7 min read
Matthias Veit

Resoto uses a directed graph to represent your infrastructure resources as nodes and relationships between them as edges. A load balancer for example is represented as node with edges pointing to all target compute instances. The compute instance might have a volume attached, where we would see an edge between the instance node and the volume node.

The nodes represent the actual resources. The edges define the relationship between the nodes. It is possible and highly likely, that one resource has multiple relationships to other resources.

Sheep Jumping on a Graph

· 2 min read
Matthias Veit

Resoto has a command-line interface which is accessible via Resoto Shell. Commands are not executed locally, but interpreted on the server. As such, only Resoto Shell is required client-side.

Resoto offers commands like echo, tail, and jq—old friends to veteran shell users—but these commands only account for a small fraction of the possibilities in Resoto's command-line interface.

Version 2.X of Resoto Shell introduces tab completion, making the command-line interface easier to use than ever before! Press the tab key, and Resoto Shell will present you with a list of available commands:

List of commands

· 4 min read
Matthias Veit

Retrieving information about resources you have deployed in your Amazon Web Services (AWS) infrastructure means tediously navigating the AWS Management Console or using the AWS Command Line Interface. This approach works well in a single account setup, but best practice is to set up a multi-account environment. And as the number of accounts grows, navigating your infrastructure and finding resources via the Console or the CLI becomes increasingly difficult.

Furthermore, the relationships between your resources are also relevant: an EBS volume is mounted to an EC2 instance running in a VPC and reachable via an ALB load balancer, for example. Developers create resources using tools such as Terraform, CDK, or CloudFormation… or sometimes even the console or CLI. How can you see everything that is running in your cloud?

Left: Sheep Spinning Up Cloud Resources; Right: Confused Sheep with Abacus

· 3 min read
Doris Houng

This past month has been quite the roller coaster for us at Some Engineering. 🎢

Right before Christmas, we decided to part ways with the name "Cloudkeeper." There were many reasons behind this decision, one of which was that we did not feel the product we were developing should be limited to cloud infrastructure. We were building a product that did much more than just "housekeeping for your clouds."

Resoto Logo Graffiti