As you might already know, managing the cost of your AWS resources is a project in itself. Although cost management, like every other AWS service, is continually evolving & improving, it still demands time & attention.

If you’ve been in the cloud for some time, you’ve developed your own ways of keeping an eye on the cost of your cloud resources, be it something as simple as switching from on-demand to reserved instances, or something as complex as running large-scale, automated analysis jobs on the AWS-provided Cost & Usage Reports (CUR) in a Redshift cluster!

Essentially, once you’re all-in on AWS, you have a pretty good sense of what to expect in your monthly AWS bills. The focus then shifts from cost control & cost optimization to just keeping an eye on the bill for anything out of the ordinary.

You expect your invoice from AWS to read the amount X ± 5-10%. The only time it should deviate from this is when you expect it to. Maybe you launched a new workload into your account & expect a 30% cost increase, or maybe you decommissioned a workload & are expecting a drop in your bill. Either way, the one thing you never want is a shock at the end of the month when you look at your bill!

To this end, people & organizations have invented many innovative solutions to always keep track of their AWS cost as it accrues over the course of a month. It was just a matter of time before AWS noticed this & introduced a fully-managed AWS service to take care of it! 😊

Enter AWS Cost Anomaly Detection…

AWS Cost Anomaly Detection

AWS Cost Anomaly Detection does exactly what it says on the box; it detects anomalies in the incurred cost of your AWS resources. How does it do that? Machine learning of course. 😀 If there is one thing that machine learning algorithms are really good at, it’s recognizing trends & patterns in data & predicting the future trend. The moment the actual trend deviates from the predicted trend, there’s an anomaly!

AWS Cost Anomaly Detection is a feature of AWS Cost Management alongside AWS Cost Explorer, savings plans & reservations. It continuously monitors your AWS cost & usage to detect unusual spending.

How it Works

The first step to using Cost Anomaly Detection is creating something called a cost monitor. Cost monitors are of 4 types:

An “AWS Services” cost monitor monitors every AWS service you use separately. It can thus detect much smaller anomalies compared to the other types. For example, if someone launched a large EC2 instance, you’ll get an alert when it causes a cost spike at the EC2 service level.

A “Linked Account” cost monitor monitors the cost at an account level. You can configure it to monitor multiple related accounts. This makes perfect sense if you have a group of accounts under an AWS Organization serving a common purpose, like hosting different environments of a workload.

The “Cost Category” & “Cost Allocation Tag” types of cost monitors monitor resources grouped by the cost category or the cost allocation tag.

Once you have a cost monitor, you can create an alert subscription to get notified of anomalies as they occur.

Alert subscriptions fall into two distinct categories:

• Ones that send out alerts for every anomaly detected as soon as it’s detected. These can only publish to SNS topics.
• Ones that send emails summarizing all anomalies that occurred in a day or week. These can only send emails.

And that’s it! That’s all it takes to start using Cost Anomaly Detection. You should start getting anomaly alerts/emails soon. You can also see a list of all anomalies detected in the past 90 days in the AWS console.

Cost Anomaly Detection: Highlights

AWS Cost Anomaly Detection stands out from any home-grown solution in a few different ways:

• Since machine learning is involved, it can adapt to expected spikes in usage, like Monday mornings or weekends.
• An alert from Cost Anomaly Detection lets you drill into the exact root cause of the cost anomaly: which account, which region & which service caused the anomaly.

Caveats

Note that the cost monitor you create is not monitoring your account in real-time! It runs as a scheduled job roughly 3 times a day after the normal AWS billing data processing job is done. As a result, you won’t get instant alerts, they can be delayed by up to a few hours. You might have accrued additional costs in the time between when the anomaly really occurred & when you were notified of it!

Conclusion

AWS Cost Anomaly Detection is the kind of service that everyone should enable. I hope this article has given you a good grasp of the concepts of the service & how it works. You can read more about it in the official AWS documentation.

About the Author ✍🏻

Harish KM is a Principal DevOps Engineer at QloudX & a top-ranked AWS Ambassador since 2020. 👨🏻‍💻

With over a decade of industry experience as everything from a full-stack engineer to a cloud architect, Harish has built many world-class solutions for clients around the world! 👷🏻‍♂️

With over 20 certifications in cloud (AWS, Azure, GCP), containers (Kubernetes, Docker) & DevOps (Terraform, Ansible, Jenkins), Harish is an expert in a multitude of technologies. 📚

These days, his focus is on the fascinating world of DevOps & how it can transform the way we do things! 🚀