How to Collect AWS Resource Data

The Amazon Web Services (AWS) collector is configured within the Resoto Worker configuration via the config command in Resoto Shell.

Prerequisites

This guide assumes that you have already installed Resoto.

Directions

1. Enable the AWS Collector

Open the Resoto Worker configuration via the config command in Resoto Shell:
```
> config edit resoto.worker
```

Add aws to the list of collectors by modifying the configuration as follows:

resotoworker:
  ...
  # List of collectors to run
  collector:
    - 'aws'
    ...
...

2. Authenticate with AWS

Resoto supports the authentication mechanisms described in the Boto3 SDK documentation. You can authenticate with AWS via an instance profile, an access key, or profiles. These credentials can be defined in the Resoto Worker configuration or as environment variables.

Resoto Worker Configuration
Environment Variables

Instance Profile
Access Key
Profiles

Configure an instance profile.
Open the Resoto Worker configuration via the config command in Resoto Shell:
```
> config edit resoto.worker
```
Add the contents of your credentials file to the resotoworker section of the configuration as follows:
Resoto Worker configuration
```
resotoworker:
  ...
  write_files_to_home_dir:
    - path: ~/.aws/credentials
      content: |
        [default]
        region = us-west-2

        role_arn = arn:aws:iam::235059640852:role/Resoto
        external_id = a5eMybsyGIowimdZqpZWxxxxxxxxxxxx
        credential_source = Ec2InstanceMetadata
...
```
note
If you do not wish to save the contents of your credentials file to Resoto's database, you can alternatively mount the ~/.aws directory to the resotoworker container.
info
Since Resoto is running on your local machine, it can access the credentials file directly at ~/.aws/credentials.

Modify the aws section of the configuration as follows, making sure that aws.access_key_id and aws.secret_access_key are set to null:

Resoto Worker configuration
resotoworker:
  ...
...
aws:
  # AWS Access Key ID (null to load from env - recommended)
  access_key_id: null
  # AWS Secret Access Key (null to load from env - recommended)
  secret_access_key: null
  ...

note

Access keys in the configuration are visible to anyone with access to your Resoto instance. You can alternatively define the access key via environment variables.

Open the Resoto Worker configuration via the config command in Resoto Shell:
```
> config edit resoto.worker
```

Modify the aws section of the configuration as follows:

Resoto Worker configuration
resotoworker:
  ...
...
aws:
  # AWS Access Key ID (null to load from env - recommended)
  access_key_id: 'AKIAZGZKXXXXXXXXXXXX'
  # AWS Secret Access Key (null to load from env - recommended)
  secret_access_key: 'vO51EW/8ILMGrSBV/Ia9Fov6xZnKxxxxxxxxxxxx'
  ...

Create a file ~/.aws/credentials with the desired profiles:
~/.aws/credentials
```
[production]
aws_xxx = yyy

[test]
aws_xxx = yyy

[dev]
aws_xxx = yyy

...
```
Open the Resoto Worker configuration via the config command in Resoto Shell:
```
> config edit resoto.worker
```
Add the contents of your credentials file to the resotoworker section of the configuration as follows:
Resoto Worker configuration
```
resotoworker:
  ...
  write_files_to_home_dir:
    - path: ~/.aws/credentials
      content: |
        [default]
        region = us-west-2

        role_arn = arn:aws:iam::235059640852:role/Resoto
        external_id = a5eMybsyGIowimdZqpZWxxxxxxxxxxxx
        credential_source = Ec2InstanceMetadata
...
```
note
If you do not wish to save the contents of your credentials file to Resoto's database, you can alternatively mount the ~/.aws directory to the resotoworker container.
info
Since Resoto is running on your local machine, it can access the credentials file directly at ~/.aws/credentials.
Modify the aws section of the configuration as follows, adding one or more profile names from your ~/.aws/credentials file:
Resoto Worker configuration
```
resotoworker:
  ...
...
aws:
  ...
  profiles:
    - production
    - test
    - dev
```
Profiles can be combined with other AWS options, such as aws.role and aws.scrape_org.
note
When switching from profiles to another authentication option, be sure to set the value of aws.profiles as null.

note

Resoto is meant to run unattended on a server using a service account or instance profile. Resoto supports the same environment variables that the AWS Command-Line Interface does (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_SESSION_TOKEN, AWS_ROLE_ARN, AWS_WEB_IDENTITY_TOKEN_FILE, AWS_ROLE_SESSION_NAME, etc.).

When using temporary credentials, however, they should be written to the credentials or config file and updated out-of-band instead of using environment variables. This is because the resotoworker process starts once and then runs forever (i.e., updated environment variables are only reflected upon restart).

You can specify a profile using AWS_PROFILE and, for local testing, SSO authentication would work as well. However, when Resoto is running unattended in a production environment, SSO credentials that require opening a browser window would not work.

Set the required environment variables (e.g., AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY):

Docker
Kubernetes
pip

Add environment variable definitions to the resotoworker service in docker-compose.yaml:

docker-compose.yaml
services:
  ...
  resotoworker:
    environment:
      - AWS_ACCESS_KEY_ID=AKIAZGZKXXXXXXXXXXXX
      - AWS_SECRET_ACCESS_KEY=vO51EW/8ILMGrSBV/Ia9Fov6xZnKxxxxxxxxxxxx
  ...
...

Recreate the resotoworker container with the updated service definition:
```
$ docker-compose up -d
```
note
Docker Compose V2 integrated compose functions in to the Docker platform.
In Docker Compose V2, the command is docker compose (no hyphen) instead of docker-compose.

Create a secret:

$ kubectl -n resoto create secret generic resoto-auth \
  --from-literal=AWS_ACCESS_KEY_ID=AKIAZGZKXXXXXXXXXXXX \
  --from-literal=AWS_SECRET_ACCESS_KEY=vO51EW/8ILMGrSBV/Ia9Fov6xZnKxxxxxxxxxxxx

Update resoto-values.yaml as follows:

resoto-values.yaml
...
resotoworker:
  ...
  extraEnv:
    - name: AWS_ACCESS_KEY_ID
      valueFrom:
        secretKeyRef:
          name: resoto-auth
          key: AWS_ACCESS_KEY_ID
    - name: AWS_SECRET_ACCESS_KEY
      valueFrom:
        secretKeyRef:
          name: resoto-auth
          key: AWS_SECRET_ACCESS_KEY
...

Deploy these changes with Helm:

$ helm upgrade resoto resoto/resoto --set image.tag=edge -f resoto-values.yaml

Export the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables:

$ export AWS_ACCESS_KEY_ID=AKIAZGZKXXXXXXXXXXXX
$ export AWS_SECRET_ACCESS_KEY=vO51EW/8ILMGrSBV/Ia9Fov6xZnKxxxxxxxxxxxx

Restart the resotoworker process.

Open the Resoto Worker configuration via the config command in Resoto Shell:
```
> config edit resoto.worker
```

Modify the aws section of the configuration as follows, making sure that aws.access_key_id and aws.secret_access_key are set to null:

Resoto Worker configuration
resotoworker:
  ...
...
aws:
  # AWS Access Key ID (null to load from env - recommended)
  access_key_id: null
  # AWS Secret Access Key (null to load from env - recommended)
  secret_access_key: null
  ...

3. Authorize Resoto Access to AWS

See How to Roll Out Resoto AWS Permissions with CloudFormation for a step-by-step guide on how to roll out Resoto permissions organization-wide.

info

If you prefer to deploy the role yourself, the S3 URL of the template to create the ResotoAccess role is https://resotopublic.s3.amazonaws.com/cf/resoto-role.template.

4. Trigger Resource Collection

By default, Resoto performs resource collection each hour. To immediately trigger a collect run, use the workflow run command in Resoto Shell:
```
> workflow run collect
```
Once the collect run completes, you can view a summary of collected AWS resources using the following search:
```
> search is(aws_resource) | count kind
```

How to Collect AWS Resource Data

Prerequisites​

Directions​

1. Enable the AWS Collector​

2. Authenticate with AWS​

3. Authorize Resoto Access to AWS​

4. Trigger Resource Collection​

Contact Us