Secrets Management on AWS: Secrets Manager vs SSM Parameter Store, Rotation, and Killing Hardcoded Credentials

Every credential your application reads at runtime is a liability you have to govern: where it lives, who can decrypt it, how often it rotates, and whether you can prove who touched it. The two AWS-native answers are Secrets Manager and SSM Parameter Store (SecureString). They overlap enough to confuse, but they differ sharply on rotation, cost, size, and cross-account behavior. This piece is for engineers who already know what a secret is and want the precise trade-offs, the IAM and KMS plumbing that makes retrieval least-privilege, and the CloudTrail signal that tells you when something reads a secret it shouldn't.

Secrets Manager vs Parameter Store: the real trade-offs

Both encrypt at rest with KMS and both gate access with IAM. The decision usually comes down to whether you need managed rotation and resource policies, versus whether you want the cheapest possible store for config that happens to be sensitive. Parameter Store SecureString parameters are free at the Standard tier (up to 4 KB); Secrets Manager bills per secret per month plus per 10,000 API calls. Secrets Manager is the right default for database and third-party API credentials that must rotate; Parameter Store is excellent for hierarchical config and SecureStrings you rotate yourself.

• Rotation: Secrets Manager has built-in, Lambda-driven automatic rotation with versioning (AWSCURRENT / AWSPENDING / AWSPREVIOUS staging labels). Parameter Store has none natively; you script it or trigger a Lambda on a schedule.
• Cost: Parameter Store Standard tier is free for storage and standard-throughput API calls. Secrets Manager charges per secret/month and per API call batch. At thousands of low-churn config values, Parameter Store wins decisively.
• Size and type: Parameter Store Standard caps at 4 KB (Advanced tier and Secrets Manager allow larger payloads, up to 8 KB / 64 KB respectively). Secrets Manager stores arbitrary JSON blobs and is built for structured credential documents.
• Cross-account and policies: Secrets Manager supports resource-based policies, so you can share a secret to another account directly. Parameter Store has no resource policy; cross-account sharing means assuming a role in the owning account.
• Replication: Secrets Manager offers built-in multi-Region replication of a secret. Parameter Store does not replicate for you.
• Integrations: Secrets Manager wires directly into RDS, Redshift, and DocumentDB managed rotation. Parameter Store integrates broadly as a config source (CloudFormation dynamic references, ECS, Lambda extensions).

Rule of thumb: if a value must rotate on a schedule or be shared cross-account by policy, use Secrets Manager. If it is sensitive config you control and rarely change, a Parameter Store SecureString is cheaper and just as encrypted. Mixing both is normal and correct.

KMS is doing the actual encryption, so own the key

Neither service encrypts anything itself; both call KMS. Secrets Manager encrypts with a KMS key (the AWS-managed aws/secretsmanager key by default), and Parameter Store SecureStrings use aws/ssm unless you specify a customer-managed key (CMK). Use a CMK. A CMK lets you write a key policy that scopes kms:Decrypt to specific principals, enables an independent audit trail, and gives you the kill switch of disabling the key. Crucially, a caller needs BOTH the secretsmanager:GetSecretValue (or ssm:GetParameter) permission AND kms:Decrypt on the encrypting key. That double gate is the foundation of least-privilege retrieval, so do not undermine it with a wildcard KMS grant.

Least-privilege retrieval: scope to one ARN and one key

The most common mistake is an identity policy that allows secretsmanager:GetSecretValue on Resource: *. That hands any compromised task the keys to every secret in the account. Scope the permission to the exact secret ARN, and add a KMS condition so the grant only works through your intended key. Note the trailing wildcard on the secret ARN: Secrets Manager appends a random six-character suffix to every secret, so an exact-prefix match with -?????? (or a trailing *) is required.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "ReadOneSecret",
      "Effect": "Allow",
      "Action": [
        "secretsmanager:GetSecretValue",
        "secretsmanager:DescribeSecret"
      ],
      "Resource": "arn:aws:secretsmanager:us-east-1:111122223333:secret:prod/api/payments-gw-??????"
    },
    {
      "Sid": "DecryptWithThatKeyOnly",
      "Effect": "Allow",
      "Action": "kms:Decrypt",
      "Resource": "arn:aws:kms:us-east-1:111122223333:key/9f3c0a4e-2b1d-4c8a-bb7e-7d6f5a4c3b2a",
      "Condition": {
        "StringEquals": {
          "kms:ViaService": "secretsmanager.us-east-1.amazonaws.com"
        }
      }
    }
  ]
}

The kms:ViaService condition means this Decrypt permission only works when the call comes through Secrets Manager, not if the identity tries to call KMS directly against arbitrary ciphertext. Pair the identity policy above with a resource policy on the secret itself for defense in depth, locking the secret to a specific role and denying anything outside your account or org. Combine that with aws:PrincipalOrgID and a deny on non-TLS calls for a hardened posture.

Automatic rotation with Lambda

Secrets Manager rotation is a four-step Lambda contract invoked with the secret ARN, a client request token, and a Step: createSecret generates a new value and stores it as AWSPENDING; setSecret writes that new credential into the target system (the database, the API provider); testSecret verifies the AWSPENDING value actually works; finishSecret promotes AWSPENDING to AWSCURRENT by moving the staging label. For RDS, Aurora, Redshift, and DocumentDB you can use the AWS-provided rotation templates and never write the Lambda yourself. The single-user versus alternating-users strategy matters: alternating-users (two rotating credentials) gives zero-downtime rotation because the old credential stays valid through the swap, which is what you want for hot production databases. Set RotationRules with a ScheduleExpression cron and let the AWSPREVIOUS label cover in-flight connections.

Retrieving a secret at runtime, the right way

Never read a secret at build time and bake it into an artifact. Fetch it at runtime with the instance/task role, cache it in memory for the credential's lifetime, and handle rotation by catching auth failures and re-fetching. The example below pulls a JSON secret and caches it; in Lambda or ECS, prefer the AWS Parameters and Secrets Lambda Extension / Secrets Manager agent, which adds a local cache layer and cuts both latency and API cost.

import json
import boto3
from botocore.exceptions import ClientError

_client = boto3.client("secretsmanager", region_name="us-east-1")
_cache = {}

def get_secret(secret_id: str) -> dict:
    if secret_id in _cache:
        return _cache[secret_id]
    try:
        resp = _client.get_secret_value(SecretId=secret_id)
    except ClientError as e:
        # AccessDeniedException here often means the KMS key, not the secret
        raise RuntimeError(f"could not fetch {secret_id}: {e}") from e
    value = json.loads(resp["SecretString"])
    _cache[secret_id] = value
    return value

# usage: creds = get_secret("prod/api/payments-gw")

Kill hardcoded credentials at the source

All of the above is wasted if a long-lived key is sitting in a Git history, a Dockerfile ENV, or an AMI. The fix is structural, not a one-time grep.

• Never bake secrets into AMIs, container images, env vars committed to source, user-data scripts, or CI config. Inject at runtime from Secrets Manager or Parameter Store via the instance/task role.
• Prefer roles over keys: EC2 instance profiles, ECS task roles, EKS IRSA / Pod Identity, and Lambda execution roles give short-lived, auto-rotated STS credentials, so there is no static key to leak.
• Enable push-protection secret scanning in your Git host and run a scanner (such as the open-source gitleaks or trufflehog) as a pre-commit hook and CI gate to block commits containing high-entropy strings or known credential patterns.
• Treat any leaked credential as compromised the instant it hits a remote: rotate or revoke it, do not just delete the commit. Git history and forks keep the old value alive.
• Scan IaC and images too, not just app code. Terraform state, Helm values, and image layers are common leak vectors.

Detect secret access in CloudTrail

Every read is logged. Secrets Manager GetSecretValue and SSM GetParameter / GetParameters (with WithDecryption=true) emit CloudTrail management events, and the corresponding KMS Decrypt call shows the encryption context, including the secret ARN. The high-value signals: GetSecretValue from a principal or role that has no business reading that secret, a sudden spike in retrievals (possible exfiltration), reads from an unexpected source IP or Region, and AccessDenied events on a secret (someone probing). Build alerts on the eventName plus the requestParameters.secretId, and join against KMS Decrypt events where the encryption context SecretARN matches. Route these to an alarm so a human sees anomalous secret access in minutes, not at the next audit.

None of these controls is exotic. The discipline is in combining them: a CMK you own, an identity policy scoped to one ARN gated by kms:ViaService, a resource policy for cross-account, managed rotation for anything with a schedule, runtime-only injection through roles, scanning that blocks leaks before they land, and CloudTrail alerting that turns every secret read into an observable event. Do that and a hardcoded credential becomes something your pipeline rejects, not something an attacker finds.