Netsil collectors can automatically detect and configure popular integrations such as Kubernetes, Consul, Redis, Memcached, Apache, etc. The Autoconf feature works by detecting specific container images and then dynamically associating the correct configurations for that image.
For example, the collector would look for the
consul docker image and automatically generate the correct configuration to pull in
Default Autoconf Templates¶
Autoconf templates are configured by default for the following applications:
In a traditional non-container environment, integration configuration is like the environment in which it runs: static. The Collector reads integration configurations from disk when it starts, and as long as it’s running, it continuously runs every configured integration. The configuration files are static, and any network-related options configured within them serve to identify specific instances of a monitored service (e.g. a redis instance at 10.0.0.61:6379).
In containerized environments the ip address and port configuration of services can be very dynamic and change at runtime. With the Autoconf feature the integration configurations can adapt to the actual ip and port of the services at runtime.
The execution model for computing autoconf differs from static config files. The collector doesn’t run all the integrations all the time unlike static config; it decides which integrations to enable by inspecting all containers currently running on the same host as the Collector.
As the Collector inspects each running container, it checks if the container matches any of the container identifiers from any loaded templates. For each match, the Collector generates a static integration configuration by substituting the matching container’s IP address and port. Then it enables the integration using the static configuration.
The Collector watches for Docker events—container creation, destruction, starts, and stops—and enables, disables, and regenerates static integration configurations on such events.
Autodiscovery uses templates for integration configuration. In each template, the Collector looks for two template variables:
%%port%% to appear in place of any normally-hardcoded network options. For example: a template for the Collector’s Go Expvar integration would contain the option
expvar_url: http://%%host%%:%%port%%. For containers that have more than one IP address or exposed port, you can direct Autodiscovery to pick the right ones by using template variable indexes.
Template Source: Files (Autoconf)¶
The Collector looks for Autodiscovery templates in its
conf.d/auto_conf directory. Some integrations as listed here are enabled by default. The default templates are good for basic use cases. The configuration can be customized further e.g. extra integration options, different container identifiers, different port mapping etc. by providing custom Autoconf files.
Template Variable Indexes¶
For containers that have many IP addresses or expose many ports, you can tell Autodiscovery which ones to choose by appending an underscore to the template variable, followed by an integer, e.g.
%%port_4%%. After inspecting the container, Autodiscovery sorts the IPs and ports numerically and in ascending order, so for a container that exposes ports 80, 443, and 8443,
%%port_0%% refers to port 80. Non-indexed template variables refer to the last item in the sorted list, so in this case,
%%port%% means port 8443.
You can also add a network name suffix to the
%%host%% variable i.e.
%%host_swarm%%, etc. for containers attached to multiple networks. When
%%host%% does not have a suffix, Autodiscovery picks the container’s bridge network IP address.
docker-images attributes to identify the application and generate the correct configuration file. It uses the default docker image name such as
consul, etc. to identify the application. If you are using custom docker images, however, then you will need to update the
For example, below is the
apache.yaml template packaged with the containerized netsil-dd-agent:
docker_images: - httpd init_config: instances: - apache_status_url: http://%%host%%/server-status?auto
docker_images option. This required option lets you provide container identifiers. Autoconf will apply this template to any containers on the same host that run an
In fact, it will apply on any
httpd image. Suppose you have one container running
library/httpd:latest and another running
yourusername/httpd:v2. Autoconf will apply the above template to both containers. When it’s loading Autoconf files, Autoconf cannot distinguish between identically-named images from different sources or with different tags, and you must provide short names for container images, e.g.
If this is too limiting; if you need to apply different integration configurations to different containers running the same image, use labels to identify the containers. Label each container differently, then add each label to the
docker_images list of any template file: The
docker_images supports any kind of container identifier, not just images.