Logging .NET to AWS CloudWatch: Using Serilog

Ivan Kahl -

In the previous post in this series, we went through a low-level demonstration of logging to AWS CloudWatch to gain an understanding of how it all works. Now that we know how it works, we can make use of an existing logging library instead. This is recommended over developing your own logging implementation as it caters for potential errors and supports different log levels and configurations.

In this post, we will learn how to configure Serilog to log to AWS CloudWatch. This will allow us to add AWS CloudWatch logging to an existing application that might already use Serilog with minimal effort.

Logging .NET to AWS CloudWatch Series

Video

Prerequisites

AWS Account

You will need an active AWS account that we can use to push logs from our .NET application to AWS CloudWatch. Our volumes will be minimal so we shouldn't exceed the free tier for AWS CloudWatch.

If you don't have an AWS account yet, you can sign up for one here.

AWS Profile

Since this article is just focusing on AWS CloudWatch, we won't be configuring any AWS credentials in our .NET application. Rather, we will let the AWS SDK automatically use our AWS profile that is set up when running the AWS CLI. This simplifies the demonstrations as we don't have to fumble around with permissions and credentials.

However, please note that you should always make sure to give applications their own IAM roles/users in production with minimal permissions according to AWS best practices.

Let's Go!

With that out the way, let's get started!

1. Create a new Console application

We will be creating another .NET 6 Console application. Once that's done, we will install and set up Serilog. I will be creating this project in Visual Studio 2022, but you can also use the .NET 6 CLI if you prefer.

Create a .NET 6 Console application

2. Add the Serilog Nuget package and set up a logger

Let's quickly set up a simple Serilog instance which we can later configure to log to AWS CloudWatch. We will need to install two Nuget packages:

  • Serilog (2.10.0 at the time of writing)
  • Serilog.Sinks.Console (4.0.1 at the time of writing)
Install the Serilog and Serilog.Sinks.Console Nuget packages

Once we've installed the packages, we can create a Serilog logger that will log to the Console for now:

Program.cs

using Serilog;

using var log = new LoggerConfiguration()
    .MinimumLevel.Verbose()
    .WriteTo.Console()
    .CreateLogger();

log.Verbose("Writing introduction message...");

log.Information("Hi there! How are you?");

log.Verbose("Wrote introduction message!");

Once you've run the program you should see the following output:

You should see this output when running the program

3. Install the Serilog.Sinks.AwsCloudWatch Nuget package

To log to AWS CloudWatch, we'll need to first install a Serilog Sink (think of a Sink as a destination for logging messages in Serilog) that can log to AWS CloudWatch. Fortunately, a Nuget package already exists for this: Serilog.Sinks.AwsCloudWatch.

4. Setup the AWS CloudWatch Sink on our logger

Let's look at how we can quickly add AWS CloudWatch as a Sink on our Serilog logger:

Program.cs

using Amazon.CloudWatchLogs;
using Serilog;
using Serilog.Sinks.AwsCloudWatch;

// Create an AWS CloudWatch client (similar to Part 1 when we did it ourselves)! In this case we are
// using the AWS profile configured on my machine but if you needed to specify credentials for AWS
// you would use the following constructor: var client = new
// AmazonCloudWatchLogsClient(awsAccessKeyId, awsSecretAccessKey);
var client = new AmazonCloudWatchLogsClient();

using var log = new LoggerConfiguration()
    .MinimumLevel.Verbose()
    .WriteTo.AmazonCloudWatch(
        // The name of the log group to log to
        logGroup: "/dotnet/logging-demo/serilog",
        // A string that our log stream names should be prefixed with. We are just specifying the
        // start timestamp as the log stream prefix
        logStreamPrefix: DateTime.UtcNow.ToString("yyyyMMddHHmmssfff"),
        // The AWS CloudWatch client to use
        cloudWatchClient: client)
    .WriteTo.Console()
    .CreateLogger();

log.Verbose("Writing introduction message...");

log.Information("Hi there! How are you?");

log.Verbose("Wrote introduction message!");

You might notice a few similarities to when we set up our own logging using the AWS SDK in part 1. We must first set up our own AWS CloudWatch client and then tell Serilog how to use the client.

If you run this program, you should see the following logs appear in AWS CloudWatch:

You should see the following output in AWS CloudWatch

We have log messages!

5. (Optional) Customising the AWS CloudWatch Sink

If you don't like the JSON output, we can change that! We can also specify various other optional configuration parameters:

Program.cs

using Amazon.CloudWatchLogs;
using Serilog;
using Serilog.Formatting.Json;
using Serilog.Sinks.AwsCloudWatch;

// Create an AWS CloudWatch client (similar to Part 1 when we did it ourselves)! In this case we are
// using the AWS profile configured on my machine but if you needed to specify credentials for AWS
// you would use the following constructor: var client = new
// AmazonCloudWatchLogsClient(awsAccessKeyId, awsSecretAccessKey);
var client = new AmazonCloudWatchLogsClient();

using var log = new LoggerConfiguration()
    .MinimumLevel.Verbose()
    .WriteTo.AmazonCloudWatch(
        // The name of the log group to log to
        logGroup: "/dotnet/logging-demo/serilog",
        // A string that our log stream names should be prefixed with. We are just specifying the
        // start timestamp as the log stream prefix
        logStreamPrefix: DateTime.UtcNow.ToString("yyyyMMddHHmmssfff"),
        // (Optional) Maximum number of log events that should be sent in a batch to AWS CloudWatch
        batchSizeLimit: 100,
        // (Optional) The maximum number of log messages that are stored locally before being sent
        // to AWS Cloudwatch
        queueSizeLimit: 10000,
        // (Optional) Similar to above, except the maximum amount of time that should pass before
        // log events must be sent to AWS CloudWatch
        batchUploadPeriodInSeconds: 15,
        // (Optional) If the log group does not exists, should we try create it?
        createLogGroup: true,
        // (Optional) The number of attempts we should make when logging log events that fail
        maxRetryAttempts: 3,
        // (Optional) Specify the time that logs should be kept for in AWS CloudWatch
        logGroupRetentionPolicy: LogGroupRetentionPolicy.OneMonth,
        // (Optional) Specify a custom text formatter for the output message.
        textFormatter: new JsonFormatter(),
        // The AWS CloudWatch client to use
        cloudWatchClient: client)
    .WriteTo.Console()
    .CreateLogger();

log.Verbose("Writing introduction message...");

log.Information("Hi there! How are you?");

log.Verbose("Wrote introduction message!");

Closing

That's all it takes to configure Serilog to log to AWS CloudWatch! It's super simple and definitely a viable option for logging to AWS CloudWatch, especially if you already have an existing application that already uses Serilog.

If you would like to find out more about Serilog you can view their wiki here. You can also read more about the AWS CloudWatch Sink here.