Contributing to Collector Plugins
Collector plugins allow for importing of arbitrary resources into Resoto in graph form. The most common use case is to gather information about cloud accounts and/or resources. However, any data expressible is graph form can be collected—be it social media accounts, software dependency trees, network topology, steps for cooking your favorite food, etc.
Plugin source code lives in the
plugins directory within the
someengineering/resoto repository on GitHub. Each plugin is maintained as separate project.
The AWS collector source code lives in the plugins/aws directory within the someengineering/resoto repository on GitHub.
Pull requests should target a single plugin. Please refer to the Components contribution guide for details on how to clone the repository, set up a virtual environment, start Resoto, and submit your changes for review.
The collector plugin interface is defined as follows:
def collect(self) -> None:
"""Collects all the Cloud Resources"""
account_graph = collect_account()
The typical flow of the
collect method is:
- An instance of a
Graphclass is created and filled with cloud resources.
- The graph is merged with the plugin's internal graph.
Resource Graph Structure
Usually, the graph is structured as follows:
- The top-level node is the cloud itself. It is added by the collector plugin automatically.
- The next level is the account (e.g., an AWS account, GCP project, DigitalOcean team, etc.).
- Where applicable, the following level is the region (e.g., US-East-1, US-West-1, etc.).
- The remaining levels are cloud-specific resources.
Merging Collected Resources
Resoto uses a thin wrapper on top of NetworkX to operate on graphs. The two most used methods in the wrapper are
Keep in mind that there are two types of edges,
defaultedges represent how resources relate to each other—for example, a resource may be represented by a child node of a region node.
deleteedges define the order in which resources should be cleaned up, i.e. that an instance needs to be deleted before the volume can be deleted.
Please refer to the example collector plugin for a model of how to implement this logic.
To test a plugin, simply launch Resoto and trigger the collect action manually by executing
workflows run collect in the shell. Once the collection is complete, you execute
search (<plugin_resource_type>) to see the newly collected resources.
- Resource properties types must be globally unique. For example, if there is a property
status: strdefined by some other plugin and you add a property
status: int, the collection will fail. The reason for such behavior is that all resource properties are indexed for search.
- Try to not introduce unique property names. Only do it in event that types are not compatible with existing properties. For example, use
status: strinstead of
- Don't forget to call
sanitize(graph)before checking its properties in tests. Please refer to the DigitalOcean plugin as an example.