Our Elasticsearch integration collects and sends inventory and metrics from your Elasticsearch cluster to our platform, where you can see the health of your Elasticsearch environment. We collect metrics at the cluster, node, and index level so you can more easily find the source of any problems.
Read on to install the integration, and to see what data we collect.
Compatibility and requirements
Our integration is compatible with Elasticsearch 5.x through 7.x
If Elasticsearch is not running on Kubernetes or Amazon ECS, you must install the infrastructure agent on a host that's running Elasticsearch. Otherwise:
- If running on Kubernetes, see these requirements.
- If running on ECS, see these requirements.
Quick start
Instrument your Elasticsearch cluster quickly and send your telemetry data with guided install. Our guided install creates a customized CLI command for your environment that downloads and installs the New Relic CLI and the infrastructure agent.
Ready to get started? Click one of these button to try it out.
Our guided install uses the infrastructure agent to set up the Elasticsearch integration. Not only that, it discovers other applications and log sources running in your environment and then recommends which ones you should instrument.
The guided install works with most setups. But if it doesn't suit your needs, you can find other methods below to get started monitoring your Elasticsearch cluster.
Install and activate
To install the Elasticsearch integration, follow the instructions for your environment:
Additional notes:
- Advanced: Integrations are also available in tarball format to allow for install outside of a package manager.
- On-host integrations do not automatically update. For best results, regularly update the integration package and the infrastructure agent.
Configuration
Configure the integration
There are several ways to configure the integration, depending on how it was installed:
- If enabled via Kubernetes: see Monitor services running on Kubernetes.
- If enabled via Amazon ECS: see Monitor services running on ECS.
- If installed on-host: edit the config in the integration's YAML config file,
elasticsearch-config.yml
.
An integration's YAML-format configuration is where you can place required login credentials and configure how data is collected. Which options you change depend on your setup and preference.
The configuration file has common settings applicable to all integrations, such as interval
, timeout
, inventory_source
. To read all about these common settings, refer to our Configuration Format document.
重要
If you are still using our legacy configuration/definition files, please refer to this document for help.
Specific settings related to Elasticsearch are defined using the env
section of the configuration file. These settings control the connection to your Elasticsearch instance as well as other security settings and features. The list of valid settings is described in the next section of this document.
Elasticsearch Instance Settings
The Elasticsearch integration collects both metrics(M) and inventory(I) information. In the table, use the Applies To column for the settings available to each collection:
Setting | Description | Default | Applies to |
---|---|---|---|
HOSTNAME | Hostname or IP where Elasticsearch is running. | localhost | M/I |
LOCAL_HOSTNAME | Hostname or IP of the Elasticsearch node from which to collect inventory data. Should only be set if you do not wish to collect inventory data against localhost. | localhost | M |
PORT | Port on which Elasticsearch is listening. | 9200 | M/I |
USERNAME | Username for accessing Elasticsearch Node. | N/A | M/I |
PASSWORD | Password for the given user. | N/A | M/I |
CLUSTER_ENVIRONMENT | A way to further specify which cluster we are gathering data for, example: 'staging'. | N/A | M/I |
USE_SSL | Use SSL when communicating with the Elasticsearch node. | false | M/I |
CA_BUNDLE_FILE | Location of SSL certificate on the host. Only required if | N/A | M/I |
CA_BUNDLE_DIR | Alternative Certificate Authority bundle directory. | N/A | M/I |
SSL_ALTERNATIVE_HOSTNAME | Alternative server hostname that the integration will accept as valid for the purposes of SSL negotiation. | N/A | M/I |
TLS_INSECURE_SKIP_VERIFY | Skip the verification of the server's certificate chain and hostname. | false | M/I |
CONFIG_PATH | Path to the Elasticsearch configuration file. | /etc/elasticsearch/ | I |
TIMEOUT | Timeout for API requests, in seconds. | 30 | M/I |
COLLECT_INDICES | Signals whether to collect indices metrics or not. | true | M |
COLLECT_PRIMARIES | Signals whether to collect primaries metrics or not. | true | M |
INDICES_REGEX | Filter which indices are collected. | N/A | M |
MASTER_ONLY | Collect cluster metrics on the elected master only. | false | M |
METRICS | Set to | false | |
INVENTORY | Set to | false |
The values for these settings can be defined in several ways:
- Adding the value directly in the config file. This is the most common way.
- Replacing the values from environment variables using the
{{}}
notation. This requires infrastructure agent v1.14.0+. Read more here or see the example below. - Using secrets management. Use this to protect sensitive information, such as passwords that would be exposed in plain text on the configuration file. For more information, see Secrets management.
Labels/Custom attributes
You can further decorate your metrics using labels. Labels allow you to add key/value pair attributes to your metrics, which you can then use to query, filter, or group your metrics on.
Our default sample config file includes examples of labels; however, as they are not mandatory, you can remove, modify, or add new ones of your choice.
labels: env: production role: load_balancer
Example configurations
Find and use data
Data from this service is reported to an integration dashboard.
Elasticsearch data is attached to the following event types:
You can query this data for troubleshooting purposes or to create custom charts and dashboards.
For more on how to find and use your data, see Understand integration data.
Metric data
The Elasticsearch integration collects the following metric data attributes. Each metric name is prefixed with a category indicator and a period, such as cluster.
or shards.
.
Elasticsearch cluster metrics
These attributes are attached to the ElasticsearchClusterSample
event type:
Metric | Description |
---|---|
| The number of data nodes in the cluster. |
| The number of nodes in the cluster. |
| The Elasticsearch cluster health: |
| The number of active shards in the cluster. |
| The number of shards that are currently initializing. |
| The number of active primary shards in the cluster. |
| The number of shards that are relocating from one node to another. |
| The number of shards that are unassigned to a node. |
Elasticsearch node metrics
These attributes are attached to the ElasticsearchNodeSample
event type:
Metric | Description |
---|---|
| The number of active searches. |
| The time spent on the search fetch. |
| The estimated size of the field data circuit breaker, in bytes. |
| The estimated size of the parent circuit breaker, in bytes. |
| The estimated size of the request circuit breaker, in bytes. |
| The number of times the field data circuit breaker has tripped. |
| The number of times the parent circuit breaker has tripped. |
| The number of times the request circuit breaker has tripped. |
| The size of the id cache, in bytes. |
| The number of index flushes to disk since start. |
| The time spent flushing the index to disk. |
| Bytes available to this Java virtual machine on this file store, in bytes. |
| The total bytes read from the file store, in bytes. |
| The total bytes used for all I/O operations on the file store, in bytes. |
| The total I/O operations on the file store. |
| The total number of reads from the file store. |
| The total size of the file store, in bytes. |
| The total number of unallocated bytes in the file store, in bytes. |
| The total number of writes to the file store. |
| The total bytes written to the file store, in bytes. |
| The number of get requests currently running. |
| The number of get requests where the document existed. |
| The time spent on get requests where the document existed. |
| The number of get requests where the document was missing. |
| The time spent on get requests where the document was missing. |
| The time spent on get requests. |
| The number of get requests. |
| The number of current open HTTP connections. |
| The number of opened HTTP connections. |
| The number of documents currently being deleted from an index. |
| The number of documents currently being indexed to an index. |
| The number of documents indexed to an index. |
| The time spent deleting documents from an index. |
| The time spent indexing documents to an index. |
| The number of documents deleted from an index. |
| The number of failed indexing operations. |
| The time indexing waited due to throttling. |
| The memory used by the query cache, in bytes. |
| The number of documents across all primary shards assigned to the node. |
| The number of query cache evictions. |
| The number of query cache hits. |
| The number of query cache misses. |
| The number of ongoing recoveries for which a shard serves as a source. |
| The number of ongoing recoveries for which a shard serves as a target. |
| The total time recoveries waited due to throttling. |
| The number of request cache evictions. |
| The number of request cache hits. |
| The memory used by the request cache, in bytes. |
| The number of request cache misses. |
| The number of segments in an index shard. |
| The maximum memory used by the index writer, in bytes. |
| The memory used by doc values, in bytes. |
| The memory used by fixed bit set, in bytes. |
| The memory used by index segments, in bytes. |
| The memory used by the index writer, in bytes. |
| The memory used by norm, in bytes. |
| The memory used by the segment version map, in bytes. |
| The memory used by stored fields, in bytes. |
| The memory used by terms, in bytes. |
| The memory used by term vectors, in bytes. |
| The number of operations in the transaction log. |
| The size of the transaction log, in bytes. |
| The number of garbage collections run by the JVM. |
| The time spent on garbage collection in the JVM. |
| The number of concurrent mark & sweep GCs in the JVM. |
| The time spent on concurrent mark & sweep GCs in the JVM. |
| The number of major GCs in the JVM that collect old generation objects. |
| The time spent in major GCs in the JVM that collect old generation objects. |
| The number of minor GCs in the JVM that collects young generation objects. |
| The time spent in minor GCs in the JVM that collects young generation objects. |
| The number of parallel new GCs in the JVM. |
| The time spent on parallel new GCs in the JVM. |
| The amount of memory guaranteed to be available to the JVM heap, in bytes. |
| The maximum amount of memory that can be used by the JVM heap, in bytes. |
| The percentage of memory currently used by the JVM heap as a value between |
| The amount of memory currently used by the JVM heap, in bytes. |
| The maximum amount of memory that can be used by the old generation heap, in bytes. |
| The maximum amount of memory that can be used by the survivor space, in bytes. |
| The maximum amount of memory that can be used by the young generation heap, in bytes. |
| The amount of memory guaranteed to be available to JVM non-heap, in bytes. |
| The amount of memory currently used by the JVM non-heap, in bytes. |
| The amount of memory currently used by the old generation heap, in bytes. |
| The amount of memory currently used by the survivor space, in bytes. |
| The amount of memory currently used by the young generation heap, in bytes. |
| The number of active threads in the JVM. |
| The peak number of threads used by the JVM. |
| The number of currently active segment merges. |
| The number of documents across segments currently being merged. |
| The number of documents across all merged segments. |
| The size of all merged segments, in bytes. |
| The number of segment merges. |
| The size of the segments currently being merged, in bytes. |
| The time spent on segment merging. |
| The number of opened file descriptors associated with the current process, or |
| The number of queries. |
| The number of index refreshes. |
| The time spent on index refreshes. |
| The number of search fetches currently running. |
| The number of search fetches. |
| The size of the store, in bytes. |
| The number of queued threads in the bulk pool. |
| The number of active threads in the bulk pool. |
| The number of rejected threads in the bulk pool. |
| The number of threads in the bulk pool. |
| The number of queued threads in the fetch shard started pool. |
| The number of rejected threads in the fetch shard started pool. |
| The number of threads in the fetch shard started pool. |
| The number of active threads in the fetch shard store pool. |
| The number of queued threads in the fetch shard store pool. |
| The number of rejected threads in the fetch shard store pool. |
| The number of threads in the fetch shard store pool. |
| The number of active threads in the flush queue. |
| The number of queued threads in the flush pool. |
| The number of rejected threads in the flush pool. |
| The number of threads in the flush pool. |
| The number of active threads for force merge operations. |
| The number of queued threads for force merge operations. |
| The number of rejected threads for force merge operations. |
| The number of threads for force merge operations. |
| The number of active threads in the generic pool. |
| The number of queued threads in the generic pool. |
| The number of rejected threads in the generic pool. |
| The number of threads in the generic pool. |
| The number of active threads in the get pool. |
| The number of queued threads in the get pool. |
| The number of rejected threads in the get pool. |
| The number of threads in the get pool. |
| The number of active threads in the index pool. |
| The number of queued threads in the index pool. |
| The number of rejected threads in the index pool. |
| The number of threads in the index pool. |
| The number of active threads in the listener pool. |
| The number of queued threads in the listener pool. |
| The number of rejected threads in the listener pool. |
| The number of threads in the listener pool. |
| The number of active threads in the management pool. |
| The number of queued threads in the management pool. |
| The number of rejected threads in the management pool. |
| The number of threads in the management pool. |
| The number of active threads in the merge pool. |
| The number of queued threads in the merge pool. |
| The number of rejected threads in the merge pool. |
| The number of threads in the merge pool. |
| The number of active threads in the percolate pool. |
| The number of queued threads in the percolate pool. |
| The number of rejected threads in the percolate pool. |
| The number of threads in the percolate pool. |
| The number of active threads in the refresh pool. |
| The number of queued threads in the refresh pool. |
| The number of rejected threads in the refresh pool. |
| The number of threads in the refresh pool. |
| The number of active threads in the search pool. |
| The number of queued threads in the search pool. |
| The number of rejected threads in the search pool. |
| The number of threads in the search pool. |
| The number of active threads in the snapshot pool. |
| The number of queued threads in the snapshot pool. |
| The number of rejected threads in the snapshot pool. |
| The number of threads in the snapshot pool. |
| The number of active threads in the fetch shard started pool. |
| The number of connections opened for cluster communication. |
| The number of packets received in cluster communication. |
| The size of data received in cluster communication, in bytes. |
| The number of packets sent in cluster communication. |
| The size of data sent in cluster communication, in bytes. |
Elasticsearch common metrics
These attributes are attached to the ElasticsearchCommonSample
event type:
Metric | Description |
---|---|
| The number of documents deleted from the primary shards. |
| The number of documents in the primary shards. |
| The number of index flushes to disk from the primary shards since start. |
| The time spent flushing the index to disk from the primary shards. |
| The number of get requests on primary shards where the document existed. |
| The time spent on get requests from the primary shards where the document existed. |
| The number of get requests from the primary shards where the document was missing. |
| The time spent on get requests from the primary shards where the document was missing. |
| The number of get requests from the primary shards. |
| The number of get requests currently running on the primary shards. |
| The time spent on get requests from the primary shards. |
| The number of documents currently being deleted from an index on the primary shards. |
| The time spent deleting documents from an index on the primary shards. |
| The number of documents currently being indexed to an index on the primary shards. |
| The time spent indexing documents to an index on the primary shards. |
| The number of documents deleted from an index on the primary shards. |
| The number of documents indexed to an index on the primary shards. |
| The number of index refreshes on the primary shards. |
| The time spent on index refreshes on the primary shards. |
| The number of currently active segment merges on the primary shards. |
| The number of documents across segments currently being merged on the primary shards. |
| The number of documents across all merged segments on the primary shards. |
| The size of the segments currently being merged on the primary shards, in bytes. |
| The number of segment merges on the primary shards. |
| The size of all merged segments on the primary shards, in bytes. |
| The time spent on segment merging on the primary shards. |
| The time spent querying on the primary shards. |
| The number of queries to the primary shards. |
| The number of currently active queries on the primary shards. |
| The number of query fetches currently running on the primary shards. |
| The time spent on query fetches on the primary shards. |
| The number of query fetches on the primary shards. |
| The size of all the primary shards, in bytes. |
Elasticsearch index metrics
These attributes are attached to the ElasticsearchIndexSample
event type:
Metric | Description |
---|---|
| The number of documents in the index. |
| The number of deleted documents in the index. |
| The status of the index: |
| The number of primary shards in the index. |
| The store size of primary shards in the index. |
| The number of replica shards in the index. |
| The store size of primary and replica shards in the index, in bytes. |
Inventory data
The Elasticsearch integration captures the configuration parameters of the Elasticsearch node, as specified in the YAML config file. It also collects node configuration information from the "_nodes/_local" endpoint. The data is available on the Inventory page, under the config/elasticsearch source. For more about inventory data, see Understand integration data.
Check the source code
This integration is open source software. That means you can browse its source code and send improvements, or create your own fork and build it.
その他のヘルプ
さらに支援が必要な場合は、これらのサポートと学習リソースを確認してください:
- Explorers Hubでは、コミュニティからのサポートを受けたり、ディスカッションに参加したりすることができます。
- 当社サイトで答えを見つけて、サポートポータルの使用方法を確認してください。
- Linux、Windows、およびmacOS向けトラブルシューティングツールであるNew Relic Diagnosticsを実行してください。
- New Relicの とandドキュメント をご確認ください。