Manage data coming into New Relic

When you connect your data to New Relic, we process what we receive and apply data dropping and transformation rules. Then we count the bytes needed to represent your data in a standard format, like JSON. If you're on our New Relic One pricing plan, you're charged by the number of bytes written to our database, above and beyond the standard amount that’s free.

The Data ingestion page shows your ingest rates for a period you specify on the top-right of the Data management hub. Since 30 days ago is the default setting, but you can also set a custom date span.

The page shows your daily average GBs, and the total for the range you set. You can toggle between an overall ingest view and an account view to identify which of your accounts are sending the most data. The page also provides the current month-to-date, and the projected end-of-month total ingest rates. With this information, you can proactively drop data or turn off agents in order to manage ingest and, therefore, costs.

If you want to take a look at how we query the data, click the ellipsis icon (just above the chart) to slide out the chart query and open it in our query builder. And If you want to drill down further into your data usage, check out the sample queries in the usage docs.

Data ingestion sources

The Data ingestion page describes which of your data sources provide the most data on average and during specific data ranges. The sources are described here.

Break down data to see what's contributing to your ingest

You can inspect your data ingest to gain more information about your ingest health. This way, you'll know your baselines, and can more easily spot an anomaly like an ingest spike, as well as its source.

To break down your ingested data, start from the chart on the Data ingestion page. Think of the data source bands as the Y axis, and the dates as the X axis. Click on the band you want to inspect at the spot in the X axis that corresponds with the date you want to investigate.

A modal opens with the account, data source, and facet selected. You can do a handful of things on this page:

• Change the account, data source, or facet you want to drill down into.
• Change the date and time to investigate.
• Review the results of the query in chart form. The chart displays the top 15 results for the facet query.
• Open the NRQL query in the Query builder where you'll find additional facets that you can use.

Learn more about NRQL queries here.

How we break your ingest data down

Some of the content in the UI is variable, depending on your account. This information is intended to help you understand how we're working with your ingest data.

• The chart on the Data ingestion page shows data usage for a little longer time frame than that covered by your retention settings for each data ingest source. If you choose a date outside of your designated retention period for an ingest source, you'll get the message that there's no chart data available. Select a more recent date to fix this problem.
• If you inspect a data source for an account that has less than a terrabyte of data, we compute the ingest volume over a 24 hour period; otherwise, we compute it for a 1 hour period.
• The ingest value provided on the main Data ingestion chart will be slightly different from that reflected during inspection. This is because our facet computation is an estimate.

Set alerts for data use

Query and alert on usage data describes how to set alerts to get notified if you're nearing data ingest limits you don't want to cross. For example, you might set an alert on logs, which can stack up quickly in an active system.

Adjust your data ingest

Drop data for lower retention costs and data compliance

On ingest, we apply data dropping rules so you won't be charged for data that's not useful. Learn how to set additional data dropping rules yourself. Use NerdGraph to drop entire data types or drop data attributes from data types so they’re not written to NRDB. This enables you to focus on the data you want, reduces retention costs, and avoids writing sensitive data to the database. For dropping log data, see Drop data with drop filter rules.

Turn off agents and integrations

If you don’t need data from specific agents or integrations that you have installed, you can uninstall/delete those tools. For instructions, see the specific documentation for an agent or integration.