Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
103 changes: 62 additions & 41 deletions docs/reference/log4net-formatter.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ mapped_pages:

# log4net [log4net-formatter]

This Layout implementation formats a log4net event into a JSON representation that adheres to the Elastic Common Schema specification.
This log4net Layout implementation formats log4net events as JSON documents adhering to the Elastic Common Schema (ECS) specification.

## Installation [_installation_4]

Expand All @@ -23,7 +23,7 @@ Add a reference to the [Elastic.CommonSchema.Log4net](http://nuget.org/packages/

### Setup using configuration [_setup_using_configuration]

Specify layout type in appender’s configuration:
Specify the layout type in the appender configuration:

```xml
<log4net>
Expand All @@ -48,56 +48,46 @@ hierarchy.Root.Level = Level.All;
hierarchy.Configured = true;
```

The `Layout = new EcsLayout()` line then instructs log4net to use ECS layout. The sample above uses the console appender, but you are free to use any appender of your choice, perhaps consider using a filesystem target and [Elastic Filebeat](https://www.elastic.co/downloads/beats/filebeat) for durable and reliable ingestion.
Setting `Layout = new EcsLayout()` instructs log4net to use the ECS layout.

The example above uses the console appender, but you can use any appender of your choice.

For durable and reliable log ingestion, consider using a file appender together with [Elastic Filebeat](https://www.elastic.co/downloads/beats/filebeat) to ship the resulting log files.

## ECS Aware Properties [_ecs_aware_properties]

Any valid ECS log template properties that is available under `LogTemplateProperties.*` e.g `LogTemplateProperties.TraceId` is supported and will directly set the appropriate ECS field.
The layout supports valid ECS message template properties listed under `LogTemplateProperties.*`, such as `LogTemplateProperties.TraceId`.

These properties allow you to set ECS fields directly through log messages.

## Output [_output]

Apart from [mandatory fields](ecs://reference/ecs-guidelines.md#_general_guidelines), the output contains additional data:

* `log.origin.file.name` is taken from `LocationInformation`
* `log.origin.file.line` is taken from `LocationInformation`
* `log.origin.function` is taken from `LocationInformation`
* `event.created` is taken from timestamp
* `event.timezone` is equal to local timezone
* `host.hostname` is taken from `HostName` property
* `process.thread.id` is taken from `ThreadName` if it has numeric value
* `process.thread.name` is taken from `ThreadName` if it doesn’t have numeric value
* `service.name` is taken from entry or calling assembly
* `service.version` is taken from entry or calling assembly
* `error.message` is taken from `ExceptionObject`
* `error.type` is taken from `ExceptionObject`
* `error.stacktrace` is taken from `ExceptionObject`
* `metadata` is taken from properties. It also contains message template and arguments in case a formatted message was logged

Sample log event output (formatted for readability):
In addition to the [mandatory ECS fields](https://www.elastic.co/guide/en/ecs/current/ecs-guidelines.html#_general_guidelines), the output contains the following information:

* `labels` - Populated from `metadata` (string and boolean properties).
* `log.origin.file.name` - Populated from `LocationInformation`.
* `log.origin.file.line` - Populated from `LocationInformation`.
* `log.origin.function` - Populated from `LocationInformation`.
* `event.created` - Populated from the log event timestamp.
* `event.timezone` - Set to the local time zone.
* `host.hostname` - Populated from the `HostName` property.
* `process.thread.id` - Populated from `ThreadName` if its value is numeric.
* `process.thread.name` - Populated from `ThreadName` if its value is non-numeric.
* `service.name` - Populated from the entry assembly or calling assembly.
* `service.version` - Populated from the entry assembly or calling assembly.
* `error.message` - Populated from `ExceptionObject`.
* `error.type` - Populated from `ExceptionObject`.
* `error.stacktrace` - Populated from `ExceptionObject`.
* `metadata` - Populated from log event properties. It also contains the message template and its arguments when a formatted message is logged.

The following is an example of a log event formatted as JSON (formatted for readability):

```json
{
"@timestamp": "2022-08-28T14:06:28.5121651+02:00",
"log.level": "INFO",
"message": "Hi! Welcome to example!",
"metadata": {
"global_property": "Example",
"message_template": "{0}! Welcome to example!"
"0": "Hi"
},
"ecs": {
"version": "8.3.1"
},
"event": {
"timezone": "Central European Time",
"created": "2022-08-28T14:06:28.5121651+02:00"
},
"host": {
"hostname": "HGU780D3"
},
"ecs.version": "9.0.0",
"log": {
"logger": "Elastic.CommonSchema.Log4net.Example.Program",
"original": null,
Expand All @@ -109,16 +99,47 @@ Sample log event output (formatted for readability):
"function": "Main"
}
},
"labels": {
"MessageTemplate": "{0}! Welcome to example!",
"0": "Hi"
},
"agent": {
"type": "Elastic.CommonSchema.Log4net.Example",
"version": "1.0.0.0"
},
"event": {
"created": "2024-04-02T17:43:55.3829964+02:00",
"timezone": "W. Europe Standard Time"
},
"host": {
"os": {
"full": "Microsoft Windows 10.0.22631",
"platform": "Win32NT",
"version": "10.0.22631.0"
},
"architecture": "X64",
"hostname": "HGU780D3",
"name": "HGU780D3"
},
"process": {
"thread": {
"id": 1
}
"name": "Elastic.CommonSchema.Log4net.Example",
"pid": 39652,
"thread.id": 17,
"thread.name": ".NET Long Running Task",
"title": ""
},
"service": {
"name": "Elastic.CommonSchema.Log4net.Example",
"type": "dotnet",
"version": "1.0.0.0"
},
"user": {
"domain": "company",
"name": "user"
},
"metadata": {
"GlobalAmountProperty": 3.14
}
}
```


185 changes: 101 additions & 84 deletions docs/reference/nlog-formatter.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ mapped_pages:

# NLog layout [nlog-formatter]

This Layout implementation formats an NLog event into a JSON representation that adheres to the Elastic Common Schema specification.
This NLog Layout implementation formats NLog log events as JSON documents adhering to the Elastic Common Schema (ECS) specification.

## Installation [_installation_3]

Expand All @@ -18,22 +18,21 @@ Add a reference to the Elastic.CommonSchema.NLog package:
<PackageReference Include="Elastic.CommonSchema.NLog" Version="8.6.0" />
```


## Usage [_usage_3]

### Setup programatically [_setup_programatically]

```csharp
Layout.Register<EcsLayout>("EcsLayout"); // Register the ECS layout.
var config = new LoggingConfiguration();
var consoleTarget = new ConsoleTarget("console") { Layout = new EcsLayout() }; // Use the ECS layout.
config.AddRule(LogLevel.Debug, LogLevel.Fatal, consoleTarget);
LogManager.Configuration = config;
var logger = LogManager.GetCurrentClassLogger();
```

In the code snippet above `Layout.Register<EcsLayout>("EcsLayout")` registers the `EcsLayout` with NLog. The `Layout = new EcsLayout()` line then instructs NLog to use the registered layout. The sample above uses the console target, but you are free to use any target of your choice; perhaps consider using a filesystem target and [Elastic Filebeat](https://www.elastic.co/downloads/beats/filebeat) for durable and reliable ingestion.
The example above uses `EcsLayout` with the NLog console target, but you can use any target of your choice.

For durable and reliable log ingestion, consider using NLog's `FileTarget` together with [Elastic Filebeat](https://www.elastic.co/downloads/beats/filebeat) to ship the resulting log files.

### Setup using NLog.config [_setup_using_nlog_config]

Expand All @@ -58,110 +57,128 @@ In the code snippet above `Layout.Register<EcsLayout>("EcsLayout")` registers th
</nlog>
```



## EcsLayout Parameter Options [_ecslayout_parameter_options]

* **Metadata Options**
* *IncludeEventProperties* - Include LogEvent properties as metadata. Default: `true`
* *IncludeScopeProperties* - Include NLog Scope Context Properties as metadata. Default: `false`
* *ExcludeProperties* - Comma separated string with names which properties to exclude.
- *IncludeEventProperties* - Determines whether NLog event properties are included as metadata. Default: `true`.
- *IncludeScopeProperties* - Determines whether NLog scope context properties are included as metadata. Default: `false`.
- *ExcludeProperties* - A comma-separated list of property names to exclude from the output.

* **Event Options**
* *EventAction* -
* *EventCategory* -
* *EventId* -
* *EventKind* -
* *EventSeverity* -
- *EventAction* -
- *EventCategory* -
- *EventId* -
- *EventKind* -
- *EventSeverity* -

* **Agent Options**
* *AgentId* -
* *AgentName* -
* *AgentType* -
* *AgentVersion* -
- *AgentId* -
- *AgentName* -
- *AgentType* -
- *AgentVersion* -

* **Process Options**
* *ProcessExecutable* - Default: `${processname:FullName=true}`
* *ProcessId* - Default: `${processid}`
* *ProcessName* - Default: `${processname:FullName=false}`
* *ProcessThreadId* - Default: `${threadid}`
* *ProcessTitle* - Default: `${processinfo:MainWindowTitle}`
- *ProcessExecutable* - Default: `${processname:FullName=true}`
- *ProcessId* - Default: `${processid}`
- *ProcessName* - Default: `${processname:FullName=false}`
- *ProcessTitle* - Default: `${processinfo:MainWindowTitle}`
- *ProcessThreadId* - Default: `${threadid}`
- *ProcessThreadName* -

* **Server Options**
* *ServerAddress* -
* *ServerIp* -
* *ServerUser* - Default: `${environment-user}`
- *ServerAddress* -
- *ServerIp* -
- *ServerUser* - Default: `${environment-user}`

* **Host Options**
* *HostId* -
* *HostIp* - Default: `${local-ip:cachedSeconds=60}`
* *HostName* - Default: `${machinename}`
- *HostId* -
- *HostIp* -
- *HostName* - Default: `${machinename}`

* **Log Origin Options**
* *LogOriginCallSiteMethod* - Default: `${exception:format=method}`
* *LogOriginCallSiteFile* - Default: `${exception:format=source}`
* *LogOriginCallSiteLine* -
* **Http Options**
* *HttpRequestId* - Default: `${aspnet-trace-identifier}`
* *HttpRequestMethod* - Default: `${aspnet-request-method}`
* *HttpRequestBytes* - Default: `${aspnet-request-contentlength}`
* *HttpRequestReferrer* - Default: `${aspnet-request-referrer}`
* *HttpResponseStatusCode* - Default: `${aspnet-response-statuscode}`
* **Url Options**
* *UrlScheme* - Default: `${aspnet-request-url:IncludeScheme=true:IncludeHost=false:IncludePath=false}`
* *UrlDomain* - Default: `${aspnet-request-url:IncludeScheme=false:IncludeHost=true:IncludePath=false}`
* *UrlPath* - Default: `${aspnet-request-url:IncludeScheme=false:IncludeHost=false:IncludePath=true}`
* *UrlPort* - Default: `${aspnet-request-url:IncludeScheme=false:IncludeHost=false:IncludePath=false:IncludePort=true}`
* *UrlQuery* - Default: `${aspnet-request-url:IncludeScheme=false:IncludeHost=false:IncludePath=false:IncludeQueryString=true}`
* *UrlUserName* - Default: `${aspnet-user-identity}`
- *LogOriginCallSiteMethod* - Default: `${exception:format=method}`
- *LogOriginCallSiteFile* - Default: `${exception:format=source}`
- *LogOriginCallSiteLine* -

* **HTTP Options**
- *HttpRequestId* - Default: `${aspnet-trace-identifier}`
- *HttpRequestMethod* - Default: `${aspnet-request-method}`
- *HttpRequestBytes* - Default: `${aspnet-request-contentlength}`
- *HttpRequestReferrer* - Default: `${aspnet-request-referrer}`
- *HttpResponseStatusCode* - Default: `${aspnet-response-statuscode}`

* **URL Options**
- *UrlScheme* - Default: `${aspnet-request-url:IncludeScheme=true:IncludeHost=false:IncludePath=false}`
- *UrlDomain* - Default: `${aspnet-request-url:IncludeScheme=false:IncludeHost=true:IncludePath=false}`
- *UrlPath* - Default: `${aspnet-request-url:IncludeScheme=false:IncludeHost=false:IncludePath=true}`
- *UrlPort* - Default: `${aspnet-request-url:IncludeScheme=false:IncludeHost=false:IncludePath=false:IncludePort=true}`
- *UrlQuery* - Default: `${aspnet-request-url:IncludeScheme=false:IncludeHost=false:IncludePath=false:IncludeQueryString=true}`
- *UrlUserName* - Default: `${aspnet-user-identity}`

* **Trace Options**
* *ApmTraceId* - Default: `${ElasticApmTraceId}`
* **Transaction Options**
* *ApmTransactionId* - Default: `${ElasticApmTransactionId}` *
- *ApmTraceId* - Default: `System.Diagnostics.Activity.Current.TraceId`
- *ApmSpanId* - Default: `System.Diagnostics.Activity.Current.SpanId`

* **Transaction Options**
- *ApmTransactionId* - Default: `${ElasticApmTransactionId}`

## ECS Aware Message Templates [_ecs_aware_message_templates_3]

Additionally any valid ECS log template properties that is available under `LogTemplateProperties.*` e.g `LogTemplateProperties.TraceId` is supported and will directly set the appropriate ECS fields.
The layout also supports valid ECS message template properties listed under `LogTemplateProperties.*`, such as `LogTemplateProperties.TraceId`.

These properties allow you to set ECS fields directly through message templates.

```csharp
logger.Info("The time is {TraceId}", "my-trace-id");
```

Will override `trace.id` on the resulting ECS json document.

This overrides `trace.id` in the resulting ECS JSON document.

## Example output from EcsLayout [_example_output_from_ecslayout]

An example of the output is given below:
The following is an example of the JSON output produced by `EcsLayout` (formatted for readability):

```json
{
"@timestamp":"2020-02-20T16:07:06.7109766+11:00",
"log.level":"Info",
"message":"Info \"X\" 2.2",
"metadata":{
"value_x":"X",
"some_y":2.2
},
"ecs":{
"version":"1.4.0"
},
"event":{
"severity":6,
"timezone":"AUS Eastern Standard Time",
"created":"2020-02-20T16:07:06.7109766+11:00"
},
"host":{
"name":"LAPTOP"
},
"log":{
"logger":"Elastic.CommonSchema.NLog",
"original":"Info {ValueX} {SomeY}"
},
"process":{
"thread":{
"id":17592
},
"pid":17592,
"name":"dotnet",
"executable":"C:\\Program Files\\dotnet\\dotnet.exe"
}
"@timestamp": "2020-02-20T16:07:06.7109766+11:00",
"log.level": "Info",
"message": "Info \"X\" 2.2",
"ecs.version": "9.0.0",
"log": {
"logger": "Elastic.CommonSchema.NLog.Tests.LogTestsBase"
},
"labels": {
"ValueX": "X"
},
"agent": {
"type": "Elastic.CommonSchema.NLog",
"version": "1.6.0"
},
"event": {
"created": "2020-02-20T16:07:06.7109766+11:00",
"severity": 6,
"timezone": "Romance Standard Time"
},
"host": {
"ip": [ "127.0.0.1" ],
"name": "LOCALHOST"
},
"process": {
"executable": "C:\\Program Files\\dotnet\\dotnet.exe",
"name": "dotnet",
"pid": 17592,
"thread.id": 17592,
"title": "15.0.0.0"
},
"server": { "user": { "name": "MyUser" } },
"service": {
"name": "Elastic.CommonSchema",
"type": "dotnet",
"version": "1.6.0"
},
"metadata": {
"SomeY": 2.2
},
"MessageTemplate": "Info {ValueX} {SomeY} {NotX}"
}
```


Loading