Monitoring costs on AWS with Lambda

Achieving daily cloud computing cost reporting with Lambda

Incredibly early on when I started learning about AWS, I developed a habit of logging in to the console every day and checking my current costs and predicted costs with AWS. I had seen various warnings in the past about cloud costs and being in first year of college, I would not have been able to afford any possible mistakes made. The habit served me quite well, I can count on one hand the amount of times that AWS made a billing error in my costs for the month. Every time the issue was resolved by someone from AWS and the odd time, I even received some credits for my trouble. This almost obsessive monitoring of cost has stuck with me for the near four to five years of using AWS, sure I have my basic CloudWatch billing alarm, but I figured I could at least improve the process.

For starters, a big improvement area is just in time spent. Logging in to AWS requires going through the various layers of authentication I have set up to just see the Management Console. I am not going to change this any time soon, beyond making it better. So, if I could avoid needing to do this all the time, that would be great. While I am sure I can find a lot of my information via Cost Explorer, it would be nice to get some predictions as to where my money is being spent. Finally, getting all of this programmatically would be several wins in the form of exercising that part of my brain, utilising the various pieces of Andromeda I built so far and just having a bit of fun.

So, enter the Cost Monitoring Lambda function! I think Lambda was an obvious choice here for the job at hand. This was going to be a recurring task, doing the same thing every day and was going to be a relatively small amount of code. A Lambda function with a CloudWatch Event that is a cron trigger is perfect here. The actual code itself, for language I went with Java (it is what I know best) and utilising the CostExplorer API from the AWS SDK. Note that ironically, using this API does cost money to run, but it is quite small at around $0.01 per API call and how much you spend is based on how you implement it. In my case, I call the API three times per run for a cost of $0.03 total, across thirty days in a month (generally) is $0.90 per month. A small price to pay for salvation from manual tasks!

For getting the results to me, I am using AWS Simple Notification Service to publish to a topic that my email address is subscribed to. SNS is a brilliant service and being honest it is probably a top five favourite for me, maybe even top three. I could format the text a bit if I wanted to in the message, but I just take the direct output from the CostExplorer results, concatenate it all into a String that is the message sent to my email. I mentioned I call the API three times, the information I get in those calls includes my current costs to date, my daily cost and my estimated month end cost. I could nearly do without the daily cost, but it is nice to see a figure to how much I am spending per day.

The code once written, is ready to go, but it is now time to get it up and running with Andromeda. Using Andromeda Functions, I get the environment on AWS itself ready, with three functions for the three lifecycles in my development process. Tuning the timeout was more important here than I thought as it is a daily job, every function start is a cold start which needed to be accounted for. I then used Andromeda Build & Deploy to grab the Github Actions files I needed for Lambda which in this case is just the Gradle build and publish to S3. But with AWS CodeArtifact now being available, this may get enhanced to being a publish to that service instead.

Then it is just a case of applying the Terraform and connecting the CloudWatch schedule to the function! It was overall a fun process. It was great to see how Andromeda would work in a more real scenario and really opened my eyes to some gaps I did not see before. Overall, this will be only the first in a long line of many more Lambda based activities, so watch this space!