Lambda monitoring architecture

The New Relic Lambda monitoring stack

There are several important parts to Lambda monitoring:

1. Your function
2. The New Relic agent or SDK
3. The New Relic Lambda Extension
4. New Relic's backend
5. The New Relic One Lambda UI

Your function

Your function is the code you want to understand. You want to know when it's encountering errors, why it's slow, or how often it gets invoked.

New Relic agent or SDK

This is a library that New Relic provides for the language that your function is written in. Its job is to do the actual monitoring of your code. It measures the duration of your function invocations, notes errors that occur, records details about the source events, and your functions responses. To do this, it needs to wrap around your Lambda invocation handler function.

With a bit more work on your part, you can break your invocation into interesting segments, and tie together the interaction of your function with other functions and services, providing a holistic view of your serverless application.

New Relic Lambda Extension

This sidecar process runs inside the Lambda execution environment, alongside your code. It enhances the telemetry that the agent collects, and sends it to New Relic's back end in batches. It can also send your function's logs to New Relic.

The extension is a small application that integrates tightly with the AWS Lambda lifecycle, and works to minimize both the time it takes your telemetry to arrive at New Relic, and the impact that instrumentation has on your function's latency and throughput.

See more about our Lambda extension.

New Relic's backend

The New Relic service receives your telemetry, processes it into AwsLambdaInvocation, AwsLambdaInvocationError, Span, and custom events, and stores all that in our telemetry database: NRDB.

Lambda UI in New Relic One

Lambda functions aren't quite like traditional services, so the experience of managing them is a little different from the classic APM experience. Inside New Relic One, Lambda functions have a custom UI, which quickly surfaces the most important information about your function, and integrates closely with our logging and distributed tracing features.

Backed by NRDB and NRQL, you can also write custom dashboards and alerts for your functions.

The legacy CloudWatch path

Older integrations send telemetry in a slightly different way. Instead of passing it off to the extension, the Agent writes it out to CloudWatch as a log line.

By adding a log subscription filter to pipe your function logs into the aws-log-ingestion Lambda function, we can recover that telemetry log line, and forward it on to New Relic, along with some interesting platform telemetry.

Experience has shown that this approach has some considerable drawbacks. The AWS CloudWatch service is pretty expensive. After the free tier, log ingestion can easily eclipse the charges you pay to New Relic to ingest your telemetry. Plus, CloudWatch doesn't have a very good record when it comes to timeliness. CloudWatch log lines take many seconds, sometimes several minutes in cases of high load, to make their way to our log collection function. This delays time to glass, and fogs the view you have of your application at high load times, when an immediate, clear view of application performance is most critical.