VirtueCloud

At VirtueCloud , we’ve observed that managing certificate rotations is a common challenge many DevOps teams face. During our consultations with clients, it became clear that automating certificate expiration alerts is a crucial yet often overlooked aspect of day-to-day operations.

To address this, we propose an efficient way to streamline the process using AWS services, helping clients avoid potential disruptions and downtime. Although AWS Certificate Manager (ACM) with Private Certificate Authority (PCA) and KMS is great for signing certificates, a more straightforward approach using AWS Secrets Manager is sometimes a better fit.

Here’s a breakdown of how we did it:

Prerequisites

- AWS account

- Basic knowledge of AWS services

- AWS CLI configured (optional)

The Problem

We decided to store our certificates in AWS Secrets Manager. The challenge that followed was how to track the expiration dates and rotate certificates before they expired.

Manual tracking of expiration dates is:

- Error-prone

- Easy to forget

- Time-consuming

- Stressful (nobody wants an expired certificate in production!)

The Solution

We set up a simple, automated system using three key AWS services:

- EventBridge for scheduling

- Lambda for checking expiration dates

- SNS for sending alerts

How It Works

- Lambda periodically inspects secrets stored in Secrets Manager.

- EventBridge triggers the inspection based on a schedule.

- SNS sends alerts to the team when a certificate is nearing expiration.

The Code

Here’s a basic Lambda function to inspect certificates and send alerts:

Setting It Up

Step 1: Create an IAM Role for Lambda

Go to the IAM Console.
Click Roles > Create role.
Select AWS Service and choose Lambda.
Add the following permissions:

Name the role secret-rotation-inspector-role.

Step 2: Create an SNS Topic

Then confirm the subscription via email.

Step 3: Create Lambda Function

Go to the Lambda Console.
Click Create function and choose Author from scratch.
Name the function secret-rotation-inspector.
Use Python 3.9 runtime and assign the IAM role secret-rotation-inspector-role.
Paste the Lambda code and deploy.

Step 4: Create EventBridge Rule

1. Go to the EventBridge Console and click Rules > Create rule.

2. Configure the rule to run quarterly:

- Name: inspect-secret-quarterly

- Description: “Inspect secrets every quarter for rotation”

- Rule type: Schedule

- Cron expression: 0 9 1 1,4,7,10 ? *

3. Set the Lambda function as the target for this rule

Why Every 3 Months?

Most certificates are valid for one year, so quarterly inspections offer ample time for renewal and avoid unnecessary Lambda executions. This frequency helps catch any missed rotations.

Step 5: Test the Setup

1. Go to the Lambda Console.

2. Select your function and click the Test tab.

3. Create a test event and execute the function.

4. Check CloudWatch logs for execution results and your email for any alerts.

Customization Options

You can easily modify the Lambda function to inspect specific secrets using tags or adjust the alert threshold from 60 days to another time frame. For more flexible alerts, consider integrating with Slack or Teams via SNS.

Cost Considerations

This setup is very cost-effective, with minimal Lambda executions, only a few SNS notifications, and low CloudWatch logging costs. In most cases, the entire solution costs under a dollar per month.

Solution Comparison

While this blog post demonstrates a simple solution using Secrets Manager, AWS offers other approaches for certificate management, such as ACM Private Certificate Authority (PCA) with KMS. However, ACM PCA comes with different cost implications:

Higher costs for standard CA: Approximately $350/month per CA, which can add up for organizations with multiple certificates or environments.
Cheaper short-lived certificate mode: Available at $50/month, but it’s designed specifically for certificates with short validity periods (typically hours or days), making it less suitable for long-term certificate management.

Given these factors, the Secrets Manager solution is ideal when you need:

Simple certificate storage and rotation
Direct key access for signing operations
A cost-effective solution, usually costing just a few dollars per month
Quick implementation with minimal setup

In contrast, ACM PCA is better suited for enterprise environments that require:

Comprehensive PKI management
More frequent certificate rotations
Environments with higher security and compliance needs

Ultimately, the right solution depends on your specific requirements and the scale of your operations. Select the method that aligns with your goals, whether it’s Secrets Manager for simplicity and cost-efficiency or a more advanced setup like ACM PCA for larger, more complex environments.

Expanding the Solution

If you're interested in automating the entire rotation process, AWS Secrets Manager offers a built-in rotation feature for certain types of secrets. For custom secrets, you could extend this setup by creating an additional Lambda function that:

Generates new secrets
Updates the necessary applications
Validates the new secret is working properly
Archives the old secrets

However, implementing full secret automation is a more advanced topic — one we’ll cover in a future post!

Key Takeaways

Automating secret rotation tracking reduces human error and saves significant time. By leveraging AWS services like Lambda, EventBridge, and SNS, you can create a robust and low-cost solution to ensure certificate rotation happens on time without manual intervention.