CloudWatch lets you publish custom metrics from your applications. These are metrics that are not provided by the AWS services themselves and usually make sense from a business perspective, not an infrastructure operations perspective, like orders processed per minute or abandoned shopping carts, etc.

Traditionally, custom metrics were published to CloudWatch by applications by calling CloudWatch’s PutMetricData API, most commonly through the use of AWS SDK for the language of your choice.

With the new CloudWatch Embedded Metric Format (EMF), you can simply embed the custom metrics in the logs that your application sends to CloudWatch, and CloudWatch will automatically extract the custom metrics from the log data. You can then graph these metrics in the CloudWatch console and even set alerts and alarms on them like other out-of-the-box metrics.

This works anywhere you publish CloudWatch logs from, ie, EC2 instances, on-prem VMs, Docker/Kubernetes containers in ECS/EKS, Lambda functions, etc.

In this article, we’ll see how to use the EMF to publish custom metrics from Lambda functions. The logs you push to CloudWatch, need to be in a particular format for this to work. You can either generate this JSON format in your application code or use helper libraries available for Node.js, Python, Java, and C#.

EMF in Node.js Lambda

AWS provides the aws-embedded-metrics NPM library to generate logs in the proper format with embedded custom metrics. Simply use it in your Lambda function code to put metrics.

The Lambda function below sends a random valued custom metric every second to CloudWatch:

const { v4: uuidv4 } = require('uuid')
const sleep = require('sleep-promise')
const { createMetricsLogger, Unit } = require('aws-embedded-metrics')

exports.lambdaHandler = async () => {
  for (let i = 0; i < 100; i++) {
    const metrics = createMetricsLogger()
    metrics.putDimensions({ App: 'MyApp' })
    metrics.setProperty('RequestID', uuidv4())
    metrics.putMetric('OrderProcessingLatency',
      Math.floor(Math.random() * (100 - 10) + 10), Unit.Milliseconds)
    await metrics.flush()
    await sleep(1000)
  }
}

If you observe the logs generated by this function, you can see the metrics being sent. Here is one such log entry:

{
    "LogGroup": "cloudwatch-emf-HelloWorldFunction-hLxOnNy5FcMy",
    "ServiceName": "cloudwatch-emf-HelloWorldFunction-hLxOnNy5FcMy",
    "ServiceType": "AWS::Lambda::Function",
    "App": "MyApp",
    "RequestID": "96dc3ac4-d2c0-4a9e-9632-fc750db63cc6",
    "executionEnvironment": "AWS_Lambda_nodejs14.x",
    "memorySize": "128",
    "functionVersion": "$LATEST",
    "logStreamId": "2021/05/20/[$LATEST]d648aa819bba46a8b591f640bec42487",
    "_aws": {
        "Timestamp": 1621494739499,
        "CloudWatchMetrics": [
            {
                "Dimensions": [
                    [
                        "LogGroup",
                        "ServiceName",
                        "ServiceType",
                        "App"
                    ]
                ],
                "Metrics": [
                    {
                        "Name": "OrderProcessingLatency",
                        "Unit": "Milliseconds"
                    }
                ],
                "Namespace": "aws-embedded-metrics"
            }
        ]
    },
    "OrderProcessingLatency": 60
}

This metric with all its dimensions can be seen in the CloudWatch console under the aws-embedded-metrics namespace:

You can now treat this as any other metric and set alarms on it and include it in dashboards.

About the Author ✍🏻

Harish KM is a Principal DevOps Engineer at QloudX & a top-ranked AWS Ambassador since 2020. 👨🏻‍💻

With over a decade of industry experience as everything from a full-stack engineer to a cloud architect, Harish has built many world-class solutions for clients around the world! 👷🏻‍♂️

With over 20 certifications in cloud (AWS, Azure, GCP), containers (Kubernetes, Docker) & DevOps (Terraform, Ansible, Jenkins), Harish is an expert in a multitude of technologies. 📚

These days, his focus is on the fascinating world of DevOps & how it can transform the way we do things! 🚀