Skip to main content

Cloud Resource Tagging with Resoto

· 7 min read
Anja Freihube

Cloud tagging strategies and policies are hailed as one of the most efficient ways to keep your cloud infrastructure controllable. But are they really?

Cloud Resource Tagging​

Generally, the idea is that every piece of cloud service gets tagged (or labeled, in case of Google Cloud) by the developers or maintainers who work with it. This could be accomplished with infrastructure-as-code (IaC) tools (such as Terraform), with a command-line interface (CLI), or in the cloud UI.

Cloud Resource Tagging Policies​

Tagging policies could require that each resource needs tags identifying the owner, cost center, product, project, and/or any other metadata. By being diligent about tagging, resources can be managed via their tags and nothing gets overlooked.

Cloud Resource Tagging Challenges​

In theory, this is the correct way to manage resources; in practice, however, this hardly ever works as intended.

Each tag created is a tag that requires maintenance. Tagging policies may change over time and people can make mistakes (in AWS, for example, tag keys are case sensitive).

And, to properly use tagging on a greenfield cloud account is one thing; to retroactively apply tags to sprawling cloud infrastructure is quite another (especially when utilizing a multi-cloud strategy, where you'd need to repeat any operation over multiple interfaces).

We asked our community about their most challenging aspect of cloud tagging strategies on LinkedIn:

Results of LinkedIn survey of biggest real-life challenge with cloud tagging strategies

Over 70% of poll respondents felt that human error is the biggest issue. Be it on the console, in the UI, in templates, or in Terraform config files, tags are initially created by humans. But then, we want them to be properly understood by unforgiving machines.

As highlighted in my last blog post, going through cloud infrastructure with a CLI can be tiresome and requires contextual knowledge (because the API differs between services). This is hardly conducive to keeping tags under control.

Enforcing Cloud Resource Tagging Strategies with Resoto​

Resoto simplifies interaction with your cloud(s), and can be a powerful tool in tag maintenance.

Consider a scenario where an internal policy demands that all AWS S3 buckets and EC2 volumes carry a tag with key costcenter and value corresponding to a department or project.

Most developers know of this requirement, but the information is often conveyed by word of mouth. As a result, the case-sensitive tag is sometimes misspelled.

note

For the sake of clarity, today's blog post only covers AWS. However, Resoto treats resources from all cloud providers the same and you can easily customize the below search queries as needed.

Check for Cloud Resource Tagging Issues​

First, we need to check all resources in AWS.

Let's find the resources fall under the policy:

Search for all resources that are EC2 Volumes or S3 Buckets, and count them
> search is(aws_ec2_volume) or is(aws_s3_bucket) | count
​total matched: 314159
​total unmatched: 0

Next, you can check how many of these resources carry the correct tag by adding a condition that a tag with key costcenter must exist:

Add the condition that a tag with key 'costcenter' must exist
> search is(aws_ec2_volume) or is(aws_s3_bucket) and tags.costcenter != null | count
​total matched: 271828
​total unmatched: 0

If you already have suspicions about certain misspellings, you can check how many of these resources carry a spelling variant of the tag by varying the tag condition:

Add the condition that a tag with key 'CostCenter' must exist
> search is(aws_ec2_volume) or is(aws_s3_bucket) and tags.CostCenter != null | count
​total matched: 42331
​total unmatched: 0

Alternatively, if you don't yet have an idea about which typos might have snuck into your data, search for all tags:

Search for all tags of all resources and count how often they occur
> search is(aws_ec2_volume) or is(aws_s3_bucket) | jq '.tags | keys | .[] | {name:.}' | flatten | count /name
​Name: 123
​CostCenter: 42331
​costcenter: 271828
​name: 359632
​owner: 389374

Fix Cloud Resources with Incorrect Tags​

Now, you can create a new tag on the affected resources with the correct key and the value from the incorrect tag:

Add a new tag to resources, taking the value from the existing tag
> search is(aws_ec2_volume) or is(aws_s3_bucket) and tags.CostCenter != null | tag update costcenter {tags.CostCenter}

Then, clean up:

Delete the incorrect tag
> search is(aws_ec2_volume) or is(aws_s3_bucket) and tags.CostCenter != null | tag delete CostCenter

Double-check that all resources now carry the correct tag:

Count resources that are EC2 Volumes or S3 Buckets that don't have a 'costcenter' tag
> search is(aws_ec2_volume) or is(aws_s3_bucket) and tags.costcenter != null | count
​total matched: 314159
​total unmatched: 0

Prevent Cloud Resource Tagging Mistakes with Resoto Jobs​

Once you know the most common mistakes, it's time to set up automation. While the above process in Resoto Shell is already infinitely more comfortable than trying the same on AWS CLI, it is more of an exploratory step than an end-all solution.

Resoto allows you to define and schedule jobs with the jobs add command to further reduce this kind of toil work:

Create a job with the id 'repair_tags' that executes after the 'collect_done' event and runs the command to add a new tag to resources
> jobs add --id repair_tags --wait-for-event collect_done: search is(aws_ec2_volume) or is(aws_s3_bucket) and tags.CostCenter != null | tag update costcenter {tags.CostCenter}

Monitor for Incorrect Cloud Resource Tags​

While the constant cleaning up behind your developers is now entirely hands-off, it would be ideal to have them avoid these kinds of tagging mistakes in the first place.

In addition to a repair job, you can also set up notifications for your team to alert them of wrong tags, providing an opportunity to fix the root cause (e.g., Cloudformation templates or Terraform config files):

Create a job to send notifications of untagged resources to Discord
> jobs add --id notify_missing_tags --wait-for-event post_collect 'search is(aws_ec2_volume) or is(aws_s3_bucket) and tags.costcenter = null | discord title="Resources missing `costcenter` tag"
webhook="https://discord.com/api/webhooks/..."'

See Alerting How-To Guides for additional notification options.

Summary​

Because Resoto has a complete view over your entire cloud asset inventory and because Resoto is service-agnostic the tag handling even of millions of resources is no longer a fight against windmills but becomes a treat instead! ✨

Resoto is open source and free to use, and currently supports AWS, Google Cloud, and DigitalOcean. Install Resoto today!

Further Reading​

Contact Us

 

 

 

Some Engineering Inc.