Configuring Collectors

Configuration parameters for the collectors are passed as environment variables during the installation process, or through a configuration script.

Collection Modes

Collectors can install in one of the following modes:

Collection with local stream processing

This is the default mode. In this mode, network packets and metrics are locally processed at the collector, and only rolled-up time series metrics are shipped over the network to the AOC. The advantage of this mode is lower network bandwidth, but at the cost of higher CPU overhead. Use this mode when limited network bandwidth is available towards the AOC. In addition, this mode only requires an http(s) based interface between the collectors and AOC. The CPU overheads in this mode may be configured via OS' scheduling priority, see NETSIL_NICE_VALUE under configuration

To enable this mode set the NETSIL_ROLE variable as sp_collector

Collection only

In this mode, the collector ships metrics and network packets to the AOC for remote stream processing. This mode has lower CPU overhead at the cost of higher network bandwidth (within the VPC if stream processor is in the same VPC), so it should be used when surplus bandwidth is available from the collector to AOC.

To enable this mode set the NETSIL_ROLE variable as collector

Traffic collection mechanisms Traffic Collection Modes

Note: You may run the collectors in Collection only mode and deploy stream processors on local instances within the same VPC. This approach has the best of both modes in terms of minimum CPU overhead and minimum outgoing bandwidth from the VPC. Refer to the standalone stream processor section for details.

Configuration Parameters

Infrastructure Tags

You can assign Infrastructure Tags during the collector installation process by passing the TAGS environment variable:

TAGS = "tag1,tag_key2:tag_value2"

You may specify tags as comma-separated singletons, key-value pairs, or both.

Please read more about configuring automatic infrastructure tag collection here.

General Configuration

Listed below are some common configuration parameters for the collectors.

Parameter Required Default Choices Description
NETSIL_SP_HOST yes n/a IP or DNS address The address of your Netsil AOC instance
NETSIL_ORGANIZATION_ID yes n/a string An identifying string for users of the AOC
NETSIL_SP_LOAD_BALANCER_PORT no 2000 Port number Override the port of AOC where the metrics are sent. Netsil SaaS can accept metrics on port 443 in addition to port 2000
NETSIL_AUTOUPGRADE_COLLECTORS no yes yes or no Whether or not to autoupgrade the collectors upon new collector releases
TAGS no Not defined n/a Comma separated list of tags. Provided as single values or key/value pairs.
NETSIL_ROLE no sp_collector collector, sp_collector or sp Run only collector, collector and stream processor or just stream processor
RUN_PRESTART_SCRIPT no no yes or no The user may provide a custom script at /usr/bin/, which will be run during configuration if this variable is set to yes.
NETSIL_NICE_VALUE no 15 -20 to 19 Apply unix nice on the stream processor processes, higher value for lower priority
NETSIL_PROXY_HOST no Not defined n/a The hostname of external proxy
NETSIL_PROXY_PORT no Not defined n/a The port of external proxy
NETSIL_PROXY_USER no Not defined n/a The user name for external proxy in case of required authentication
NETSIL_PROXY_PWD no Not defined n/a The password for external proxy in case of required authentication
STATSD_PORT no 8125 Port number You may have to change this port if you have a statsd daemon running on the same machine as the collectors
DD_AGENT_LISTEN_PORT no 17130 Port number You may have to change this port if you have another dd-agent running on the same machine as the collectors
NETSIL_SP_LISTEN_PORT no 2005 Port number You may have to change this if tcp/udp port 2005 and tcp port 3005 are not available on the host. The collector listens on tcp and udp port NETSIL_SP_LISTEN_PORT and tcp port NETSIL_SP_LISTEN_PORT + 1000
NETSIL_SP_HOST_OVERRIDE no n/a IP or DNS address Override the address of Stream Processor where the captured traffic is sent
NETSIL_SP_PORT_OVERRIDE no n/a Port number Override the Stream Processor port where the captured traffic is sent
NETSIL_NETWORK_NAMESPACE no netsil string This parameter helps Netsil's discovery backend distinguish between distinct entities having the same IP address for e.g. across overlapping IP address spaces of two distinct VPC's. Also, the Collector to Stream Processor mapping is retained within the same network namespace when using standalone Stream Processor.

Remote Packet Capture Configuration

Listed below are the remote packet capture configuration parameters, which tune the traffic-capture capabilities of the collector.

Parameter Required Default Choices Description
BPFFILTER no n/a String Sets BPF filter. For custom filters, you may look at the BPF documentation
SAMPLINGRATE no 100 Integer <0 - 100> Percentage-based sampling rate -- the percentage of network flows (tcp transactions) to select for protocol parsing. 0 implies none of the flows are selected and 100 implies that all the flows are selected
ENABLE_SSLSPLIT no yes yes or no Enables ssl capture
INTERFACE no any String Sets the network interface on which to capture traffic
TCPMODE no YES YES or NO Enables TCP data channel
MTUSIZE no 1432 Integer Sets the MTU size. This should be less than or equal to the MTU size supported by the network minus 28


Netsil has a variety of integrations that bolster the monitoring capabilities of the AOC. Please check them out on the sidebar menu.