New Relic can parse common log formats according to built-in rules, so that you don't have to create your own parsing rules. Here are the log parsing rules, their Grok patterns, and what fields are parsed.
- To enable built-in log parsing, see our documentation for adding the
logtype
attribute. - To manage your parsing rules programmatically, use NerdGraph, our GraphQL-format API, at api.newrelic.com/graphiql. For more information, see the NerdGraph tutorial to create, query, and delete your parsing rules.
Source: logtype = 'apache'
Grok:
%{IPORHOST:clientip} %{USER:ident} %{USER:auth} \[%{HTTPDATE:timestamp}\] "(?:%{WORD:verb} %{NOTSPACE:request}(?: HTTP/%{NUMBER:httpversion})?|%{DATA:rawrequest})" %{NUMBER:response} (?:%{NUMBER:bytes}|-) %{QS:referrer} %{QS:agent}
Results:
clientip
: The IP address of the client.verb
: The HTTP verbident
: The user identity of the client making the requestresponse
: The HTTP status code of the responserequest
: The URI and request being madehttpversion
: The HTTP version of the requestrawrequest
: The raw HTTP request if data is postedbytes
: The number of bytes sentreferrer
: The HTTP referreragent
: The client's user agent
Source: logtype = 'alb'
Grok:
^%{NOTSPACE:type} %{TIMESTAMP_ISO8601:time} %{NOTSPACE:elb} %{NOTSPACE:client_ip}:%{NOTSPACE:client_port} ((%{NOTSPACE:target_ip}:%{NOTSPACE:target_port})|-) %{NOTSPACE:request_processing_time} %{NOTSPACE:target_processing_time} %{NOTSPACE:response_processing_time} %{NOTSPACE:elb_status_code} %{NOTSPACE:target_status_code} %{NOTSPACE:received_bytes} %{NOTSPACE:sent_bytes} "%{DATA:request}" "%{DATA:user_agent}" %{NOTSPACE:ssl_cipher} %{NOTSPACE:ssl_protocol} %{NOTSPACE:target_group_arn} "%{DATA:trace_id}" "%{NOTSPACE:domain_name}" "%{NOTSPACE:chosen_cert_arn}" %{NOTSPACE:matched_rule_priority} %{TIMESTAMP_ISO8601:request_creation_time} "%{NOTSPACE:actions_executed}" "%{NOTSPACE:redirect_url}" "%{NOTSPACE:error_reason}" (?:"|)%{DATA:target_port_list}(?:"|) (?:"|)%{DATA:target_status_code_list}(?:"|) "%{NOTSPACE:classification}" "%{NOTSPACE:classification_reason}"
Results:
Field | Definition |
---|---|
| The type of request or connection. Possible values are:
|
| The resource ID of the load balancer. If you are parsing access log entries, note that resources IDs can contain forward slashes ( |
| The IP address and port of the requesting client |
| The IP address and port of the target that processed this request.
|
| The total time elapsed (in seconds, with millisecond precision) from the time the load balancer received the request until the time it sent it to a target.
|
| The total time elapsed (in seconds, with millisecond precision) from the time the load balancer sent the request to a target until the target started to send the response headers.
|
| The total time elapsed (in seconds, with millisecond precision) from the time the load balancer received the response header from the target until it started to send the response to the client. This includes both the queuing time at the load balancer and the connection acquisition time from the load balancer to the client. This value is set to |
| The status code of the response from the load balancer |
| The status code of the response from the target. This value is recorded only if a connection was established to the target and the target sent a response. Otherwise, it is set to |
| The size of the request, in bytes, received from the client (requester). For HTTP requests, this includes the headers. For WebSockets, this is the total number of bytes received from the client on the connection. |
| The size of the response, in bytes, sent to the client (requester). For HTTP requests, this includes the headers. For WebSockets, this is the total number of bytes sent to the client on the connection. |
| The HTTP verb of the request |
| The URI the request was targeting |
| The HTTP version number of the request |
| User-Agent string that identifies the client that originated the request, enclosed in double quotes. The string consists of one or more product identifiers, product/version. If the string is longer than 8 KB, it is truncated. |
| The SSL cipher. This value is set to |
| The SSL protocol. This value is set to |
| The Amazon Resource Name (ARN) of the target group |
| The contents of the |
| The SNI domain provided by the client during the TLS handshake, enclosed in double quotes. This value is set to |
| The ARN of the certificate presented to the client, enclosed in double quotes.
|
| The priority value of the rule that matched the request.
|
| The time when the load balancer received the request from the client, in ISO 8601 format. |
| The actions taken when processing the request, enclosed in double quotes. This value is a comma-separated list that can include the values described in |
| The URL of the redirect target for the location header of the HTTP response, enclosed in double quotes. If no redirect actions were taken, this value is set to |
| The error reason code, enclosed in double quotes.
|
Source: logtype = 'cloudfront-web'
Grok:
^%{NOTSPACE:date}%{SPACE}%{NOTSPACE:time}%{SPACE}%{NOTSPACE:x_edge_location}%{SPACE}%{NOTSPACE:sc_bytes}%{SPACE}%{NOTSPACE:c_ip}%{SPACE}%{NOTSPACE:cs_method}%{SPACE}%{NOTSPACE:cs_host}%{SPACE}%{NOTSPACE:cs_uri_stem}%{SPACE}%{NOTSPACE:sc_status}%{SPACE}%{NOTSPACE:cs_referer}%{SPACE}%{NOTSPACE:cs_user_agent}%{SPACE}%{NOTSPACE:cs_uri_query}%{SPACE}%{NOTSPACE:cs_Cookie}%{SPACE}%{NOTSPACE:x_edge_result_type}%{SPACE}%{NOTSPACE:x_edge_request_id}%{SPACE}%{NOTSPACE:x_host_header}%{SPACE}%{NOTSPACE:cs_protocol}%{SPACE}%{NOTSPACE:cs_bytes}%{SPACE}%{NOTSPACE:time_taken}%{SPACE}%{NOTSPACE:x_forwarded_for}%{SPACE}%{NOTSPACE:ssl_protocol}%{SPACE}%{NOTSPACE:ssl_cipher}%{SPACE}%{NOTSPACE:x_edge_response_result_type}%{SPACE}%{NOTSPACE:cs_protocol_version}%{SPACE}%{NOTSPACE:fle_status}%{SPACE}%{NOTSPACE:fle_encrypted_fields}%{SPACE}%{NOTSPACE:c_port}%{SPACE}%{NOTSPACE:time_to_first_byte}%{SPACE}%{NOTSPACE:x_edge_detailed_result_type}%{SPACE}%{NOTSPACE:sc_content_type}%{SPACE}%{NOTSPACE:sc_content_len}%{SPACE}%{NOTSPACE:sc_range_start}%{SPACE}%{NOTSPACE:sc_range_end}
Results:
Field | Definition |
---|---|
| The edge location that served the request. Each edge location is identified by a three-letter code and an arbitrarily assigned number; for example, The three-letter code typically corresponds with the International Air Transport Association airport code for an airport near the edge location. (These abbreviations might change in the future.) |
| The total number of bytes that CloudFront served to the viewer in response to the request, including headers; for example, |
| The IP address of the viewer that made the request, either in IPv4 or IPv6 format. If the viewer used an HTTP proxy or a load balancer to send the request, the value of |
| The HTTP request method: |
| The domain name of the CloudFront distribution; for example, |
| The portion of the URI that identifies the path and object; for example, |
| An HTTP status code; for example, |
| The name of the domain that originated the request. Common referrers include search engines, other websites that link directly to your objects, and your own website. |
| The value of the User-Agent header in the request. The User-Agent header identifies the source of the request, such as the type of device and browser that submitted the request, and which search engine if applicable. |
| The query string portion of the URI, if any. When a URI doesn't contain a query string, this field's value is a hyphen ( |
| The cookie header in the request, including name-value pairs and the associated attributes.
|
| How CloudFront classifies the response after the last byte left the edge location. In some cases, the result type can change between the time that CloudFront is ready to send the response and the time that CloudFront has finished sending the response. |
| An encrypted string that uniquely identifies a request. In the response header, this is |
| The value that the viewer included in the Host header for this request. This is the domain name in the request.
|
| The protocol that the viewer specified in the request: |
| The number of bytes of data that the viewer included in the request, including headers. For WebSocket connections, this is the total number of bytes sent from the client to the server on the connection. |
| The number of seconds (to the thousandth of a second; for example, 0.002) between the time that a CloudFront edge server receives a viewer's request and the time that CloudFront writes the last byte of the response to the edge server's output queue as measured on the server. From the perspective of the viewer, the total time to get the full object will be longer than this value due to network latency and TCP buffering. |
| If the viewer used an HTTP proxy or a load balancer to send the request, the value of This field contains IPv4 and IPv6 addresses as applicable. If the viewer did not use an HTTP proxy or a load balancer, the value of |
| When
|
| When
|
| How CloudFront classified the response just before returning the response to the viewer. Possible values include:
|
| The HTTP version that the viewer specified in the request. Possible values include:
|
| When field-level encryption is configured for a distribution, this field contains a code that indicates whether the request body was successfully processed. If field-level encryption is not configured for the distribution, the value is a hyphen ( |
| The number of fields that CloudFront encrypted and forwarded to the origin. CloudFront streams the processed request to the origin as it encrypts data, so If field-level encryption is not configured for the distribution, the value of |
| The port number of the request from the viewer. |
| The number of seconds between receiving the request and writing the first byte of the response, as measured on the server. |
| When |
| The value of the HTTP Content-Type header of the response. |
| The value of the HTTP Content-Length header of the response. |
| When the response contains the HTTP Content-Range header, this field contains the range start value. |
| When the response contains the HTTP Content-Range header, this field contains the range end value. |
Source: logtype = 'elb'
Grok:
^%{TIMESTAMP_ISO8601:time} %{NOTSPACE:elb} %{NOTSPACE:client_ip}:%{NOTSPACE:client_port} ((%{NOTSPACE:backend_ip}:%{NOTSPACE:backend_port})|-) %{NOTSPACE:request_processing_time} %{NOTSPACE:backend_processing_time} %{NOTSPACE:response_processing_time} %{NOTSPACE:elb_status_code} %{NOTSPACE:backend_status_code} %{NOTSPACE:received_bytes} %{NOTSPACE:sent_bytes} "%{DATA:request}" "%{DATA:user_agent}" %{NOTSPACE:ssl_cipher} %{NOTSPACE:ssl_protocol}
Results:
Field | Definition |
---|---|
| The edge location that served the request. Each edge location is identified by a three-letter code and an arbitrarily assigned number; for example, |
| The total number of bytes that CloudFront served to the viewer in response to the request, including headers; for example, |
| The IP address of the viewer that made the request. If the viewer used an HTTP proxy or a load balancer to send the request, the value of |
| The HTTP request method: |
| The domain name of the CloudFront distribution; for example, |
| The portion of the URI that identifies the path and object; for example, |
| An HTTP status code (for example, If the viewer closes the connection after CloudFront starts to send the response, the log contains the applicable HTTP status code. |
| he name of the domain that originated the request. Common referrers include search engines, other websites that link directly to your objects, and your own website. |
| The value of the User-Agent header in the request. The User-Agent header identifies the source of the request, such as the type of device and browser that submitted the request and which search engine if applicable. |
| The query string portion of the URI, if any. When a URI doesn't contain a query string, this field's value is a hyphen ( |
| The cookie header in the request, including name-value pairs and the associated attributes.
|
| How CloudFront classifies the response after the last byte left the edge location. In some cases, the result type can change between the time that CloudFront is ready to send the response and the time that CloudFront has finished sending the response. |
| An encrypted string that uniquely identifies a request. In the response header, this is |
| The value that the viewer included in the Host header for this request. This is the domain name in the request.
|
| The protocol that the viewer specified in the request: |
| The number of bytes of data that the viewer included in the request, including headers. For WebSocket connections, this is the total number of bytes sent from the client to the server on the connection. |
| The number of seconds (to the thousandth of a second; for example, 0.002) between the time that a CloudFront edge server receives a viewer's request and the time that CloudFront writes the last byte of the response to the edge server's output queue as measured on the server. From the perspective of the viewer, the total time to get the full object will be longer than this value due to network latency and TCP buffering. |
| If the viewer used an HTTP proxy or a load balancer to send the request, the value of If the viewer did not use an HTTP proxy or a load balancer, the value of |
| When
|
| When
|
| How CloudFront classified the response just before returning the response to the viewer. Possible values include:
|
| The HTTP version that the viewer specified in the request. Possible values include:
|
| When field-level encryption is configured for a distribution, this field contains a code that indicates whether the request body was successfully processed. If field-level encryption is not configured for the distribution, the value of this field is a hyphen ( When CloudFront successfully processes the request body, encrypts values in the specified fields, and forwards the request to the origin, the value of this field is If the request exceeds a field-level encryption limit, |
| The number of fields that CloudFront encrypted and forwarded to the origin. CloudFront streams the processed request to the origin as it encrypts data, so If field-level encryption is not configured for the distribution, the value of |
| The port number of the request from the viewer. |
| The number of seconds between receiving the request and writing the first byte of the response, as measured on the server. |
| When |
| The value of the HTTP Content-Type header of the response. |
| The value of the HTTP Content-Length header of the response. |
| When the response contains the HTTP Content-Range header, this field contains the range start value. |
| When the response contains the HTTP Content-Range header, this field contains the range end value. |
Source: logtype = 'iis_w3c'
Grok:
%{TIMESTAMP_ISO8601:log_timestamp} %{NOTSPACE:server_ip} %{WORD:method} %{NOTSPACE:uri} %{NOTSPACE:uri_query} %{NOTSPACE:server_port} %{NOTSPACE:username} %{NOTSPACE:client_ip} %{NOTSPACE:user_agent} %{NOTSPACE:referer} %{NOTSPACE:status} %{NOTSPACE:substatus} %{NOTSPACE:win32_status} %{NOTSPACE:time_taken}
Source: logtype = 'monit'
Grok:
\\[%{NOTSPACE:tz} %{SYSLOGTIMESTAMP:nr_timestamp}\\] %{WORD:state}%{SPACE}: %{GREEDYDATA:message}
Results:
state
: The severity of the log linemessage
: The message
Source: logtype = 'mysql-error'
Grok:
\\[%{WORD:log_level}\\]
Results:
log_level
: The severity of the log line
Source: logtype = 'nginx'
Grok:
%{IPORHOST:clientip} %{USER:ident} %{USER:auth} \[%{HTTPDATE:timestamp}\] "(?:%{WORD:verb} %{NOTSPACE:request}(?: HTTP/%{NUMBER:httpversion})?|%{DATA:rawrequest})" %{NUMBER:response} (?:%{NUMBER:bytes}|-) %{QS:referrer} %{QS:agent}
Results:
clientip
: The IP address of the clientverb
: The HTTP verbident
: The user identity of the client making the requestresponse
: The HTTP status code of the responserequest
: The URI and request being madehttpversion
: The HTTP version of the requestrawrequest
: The raw HTTP request if data is postedbytes
: The number of bytes sentreferrer
: The HTTP referreragent
: The client's user agent
Source: logtype = 'nginx-error'
Grok:
^(?<timestamp>%{YEAR:year}[./-]%{MONTHNUM:month}[./-]%{MONTHDAY:day}[- ]%{TIME:time}) \\[%{LOGLEVEL:severity}\\] %{POSINT:pid}#%{NUMBER}: %{GREEDYDATA:errormessage}(?:, client: (?<client>%{IP:clientip}|%{HOSTNAME:hostname}))(?:, server: %{IPORHOSTORUNDERSCORE:server})(?:, request: %{QS:request})?(?:, upstream: \"%{URI:upstream}\")?(?:, host: %{QS:host})?(?:, referrer: \"%{URI:referrer}\")?$
Results:
severity
: The severity of the log linepid
: The server process IDerrormessage
: The error messageclientip
: The IP address of the calling clientserver
: The server IP addressrequest
: The full requestupstream
: The upstream URIhost
: The server's hostnamereferrer
: The HTTP referrer
Source: logtype = 'route-53'
Grok:
%{NUMBER:log_format_version} %{TIMESTAMP_ISO8601} %{WORD:zone_id} %{IPORHOST:query} %{WORD:query_type} %{WORD:response_code} %{WORD:protocol} %{WORD:edge_location} %{IP:resolver_ip} %{GREEDYDATA:edns_client_subnet}
Results:
log_format_version
: A versioned format for the log.zone_id
: The ID of the hosted zone that is associated with all the DNS queries in this log.query
: The domain or subdomain that was specified in the request.query_type
: Either the DNS record type that was specified in the request, orANY
.response_code
: The DNS response code that Route 53 returned in response to the DNS query.protocol
: The protocol that was used to submit the query, either TCP or UDP.edge_location
: The Route 53 edge location that responded to the query. Each edge location is identified by a three-letter code and an arbitrary number; for example,DFW3
. The three-letter code typically corresponds with the International Air Transport Association airport code for an airport near the edge location. (These abbreviations might change in the future.)resolver_ip
: The IP address of the DNS resolver that submitted the request to Route 53.edns_client_subnet
: A partial IP address for the client that the request originated from, if available from the DNS resolver.
Source: logtype = 'syslog-rfc5424'
Grok:
<%{NONNEGINT:pri}>%{NONNEGINT:version} +(?:%{TIMESTAMP_ISO8601:log.timestamp}|-) +(?:%{HOSTNAME:hostname}|-) +(?:\\-|%{NOTSPACE:app.name}) +(?:\\-|%{NOTSPACE:procid}) (?:\\-|%{NOTSPACE:msgid}) +(?:\[%{DATA:structured.data}\]|-|) +%{GREEDYDATA:message}
Results:
pri
: The priority represents both the message facility and severity.version
: Syslog protocol version.log.timestamp
: Original timestamp.hostname
: The machine that originally sent the Syslog message.app.name
: The device or application that originated the message.procid
: The process name or process ID associated with a Syslog system.msgid
: Identifies the type of message.structured.data
: Structured data string value.sd.<var>sd-id</var>.<var>sd-param-name</var>
: Thestructured.data
content is also parsed into separate attributes following a predefined naming convention:sd.<var>sd-id</var>.<var>sd-param-name</var>
. See the structured data parsing examples, which follow.message
: Free-form message that provides information about the event.Structured data parsing examples:
The structured data
[example one="1" two="2"]
would be parsed into two different attributes:sd.example.one: "1"sd.example.two: "2"If the same structured data block contains duplicate param names, it also appends an index-based suffix on the attribute name. For example, the structured data
[example number="1" number="2"]
would be parsed as:sd.example.number.0: "1"sd.example.number.1: "2"For structured data with enterprise numbers assigned, an extra attribute is also parsed. For example, the structured data
[example@123 number="1"]
would be parsed as:sd.example.enterprise.number: 123sd.example.number: "1"
その他のヘルプ
さらに支援が必要な場合は、これらのサポートと学習リソースを確認してください:
- Explorers Hubでは、コミュニティからのサポートを受けたり、ディスカッションに参加したりすることができます。
- 当社サイトで答えを見つけて、サポートポータルの使用方法を確認してください。
- Linux、Windows、およびmacOS向けトラブルシューティングツールであるNew Relic Diagnosticsを実行してください。
- New Relicの とandドキュメント をご確認ください。