Cost Anomaly Detection

AWS Cost Anomaly Detection is a relatively new feature of AWS Cost Management. It continuously monitors your AWS resource usage & the cost incurred, and detects unusual spending, like a significant spike (up or down) in your bill.

How it Works

To enable this feature in your AWS account, you first create a “cost monitor.” This is the actual AWS resource that does the “cost monitoring.” You can configure a cost monitor to either look at all services in your account, multiple accounts, cost categories, or cost allocation tags. You also create an “alert subscription,” to get notified when a cost monitor detects an anomaly.

For a much more in-depth understanding of what Cost Anomaly Detection is, why it exists, how it works, and how to set it up in your account, see my earlier post:

AWS Cost Anomaly Detection: Why, What & How

Cost Anomaly Detection for Everyone

Once you understand Cost Anomaly Detection, you’ll agree that it’s the kind of service that should be turned on in every account; there’s no downside to turning it on.

To that end, we at QloudX decided to do the same for one of our large enterprise clients. Since we are dealing with 80+ AWS accounts here & every single one of them needed Cost Anomaly Detection turned on, we turned to our trusty Terraform to automate the whole thing. 😊

Terraform for Cost Anomaly Detection

As we explored more, we realized that Terraform doesn’t yet have a resource for creating cost monitors & alert subscriptions! If you’re in the same boat as us, here is the GitHub issue tracking this:

Terraform Resources for the Cost Explorer Service

Please leave a 👍 on that issue to prioritize it.

AWS CLI to the Rescue

So, how can you create an AWS resource that does not have a corresponding Terraform resource? One way is to run the AWS CLI from within Terraform. The traditional way of doing this has been to use a local-exec provisioner inside a null_resource resource:

resource "null_resource" "cost_anomaly_monitor" {
  provisioner "local-exec" {
    command = <<CMD
      aws s3 ls <= some AWS CLI command here
    CMD
  }
}

Terraform Provisioners

Provisioners in Terraform are blocks of Terraform configuration that you can use to perform specific actions on a local or remote machine to prepare it for service.

Provisioners are a last resort. You should use them only when it’s impossible to create your desired resource using Terraform’s declarative configuration.

To learn more about Terraform provisioners, see:

Provisioners in Terraform

Despite being forced to fall back on provisioners, we wanted to take advantage of all the nice features Terraform provides, like state management & deleting the AWS resource when the Terraform resource is destroyed. So we came up with something a bit more sophisticated.

Terraform Resource for Cost Anomaly Monitor

resource "null_resource" "cost_anomaly_monitor" {
  provisioner "local-exec" {
    command = <<CMD
      aws ce create-anomaly-monitor --anomaly-monitor \
        '${jsonencode(local.AnomalyMonitor)}'
    CMD
  }

  provisioner "local-exec" {
    when    = destroy
    command = <<CMD
      aws ce delete-anomaly-monitor --monitor-arn \
        ${data.external.cost_anomaly_monitor.result.ARN}
    CMD
  }
}

There are two provisioners in this null resource: one runs when the resource is created, and the other one runs when the resource is destroyed. And so, the create-time provisioner runs aws ce create-anomaly-monitor & the destroy-time provisioner runs aws ce delete-anomaly-monitor.

Create Anomaly Monitor

The input for create-anomaly-monitor is a JSON object defining the name, type & dimension of the monitor. This is declared separately in a Terraform locals block:

locals {
  AnomalyMonitorName = "all-services-cost-monitor"
  AnomalyMonitor = {
    MonitorName      = local.AnomalyMonitorName
    MonitorType      = "DIMENSIONAL"
    MonitorDimension = "SERVICE"
  }
}

Delete Anomaly Monitor

The input for delete-anomaly-monitor is just the monitor ARN. Unfortunately, there is no easy way to get this ARN; you must run yet another AWS CLI command. And so we did; we ran a command inside a Terraform external data source to get the ARN & used that in the provisioner above:

data "external" "cost_anomaly_monitor" {
  program = ["aws", "ce", "get-anomaly-monitors", "--query",
  "((AnomalyMonitors[?MonitorName==`${local.AnomalyMonitorName}`].MonitorArn)[0]|{ARN:@})||{ARN:`ARN`}"]
}

As you can see, there is a pretty complex JMESPath query required to get just the ARN out of the command’s output.

Terraform Resource for Cost Anomaly Subscription

Just as we did for the cost monitor, we can write some more Terraform configuration for the cost anomaly subscription:

locals {
  AnomalySubscriptionName = "all-services-cost-anomaly-subscription"
  AnomalySubscription = {
    SubscriptionName = local.AnomalySubscriptionName
    Threshold        = var.cost_anomaly_threshold
    Frequency        = "IMMEDIATE"
    MonitorArnList   = [data.external.cost_anomaly_monitor.result.ARN]
    Subscribers = [{
      Type    = "SNS"
      Status  = "CONFIRMED"
      Address = aws_sns_topic.cost_anomaly.arn
    }]
  }
}

resource "null_resource" "cost_anomaly_subscription" {
  depends_on = [null_resource.cost_anomaly_monitor]

  provisioner "local-exec" {
    command = <<CMD
      aws ce create-anomaly-subscription --anomaly-subscription \
        '${jsonencode(local.AnomalySubscription)}'
    CMD
  }

  provisioner "local-exec" {
    when    = destroy
    command = <<CMD
      aws ce delete-anomaly-subscription --subscription-arn \
        '${data.external.cost_anomaly_subscription.result.ARN}'
    CMD
  }
}

data "external" "cost_anomaly_subscription" {
  program = ["aws", "ce", "get-anomaly-subscriptions", "--query",
  "((AnomalySubscriptions[?SubscriptionName==`${local.AnomalySubscriptionName}`].SubscriptionArn)[0]|{ARN:@})||{ARN:`ARN`}"]
}

Conclusion

Although not as easy as it should be, it is still possible to create & manage cost anomaly resources through Terraform. If you can wait for the official Terraform resources to be implemented, that’s best, but if like us, you need this now, this article describes how you can achieve it.

About the Author ✍🏻

Harish KM is a Principal DevOps Engineer at QloudX & a top-ranked AWS Ambassador since 2020. 👨🏻‍💻

With over a decade of industry experience as everything from a full-stack engineer to a cloud architect, Harish has built many world-class solutions for clients around the world! 👷🏻‍♂️

With over 20 certifications in cloud (AWS, Azure, GCP), containers (Kubernetes, Docker) & DevOps (Terraform, Ansible, Jenkins), Harish is an expert in a multitude of technologies. 📚

These days, his focus is on the fascinating world of DevOps & how it can transform the way we do things! 🚀

Cost Anomaly Detection Terraform