How to Load a Prompt From AWS SSM Parameter Store

Why hardcoded prompts slow you down

A summarizer service that keeps its prompt in the source is easy to write and painful to iterate:

PROMPT_TEMPLATE = "Summarize this text in one sentence:\n{text}"

Prompts are the part of an AI feature you change most - "make it two sentences," "use a formal tone," "add a length cap." With the template baked into the code, every one of those tweaks is a code change, a pull request, a CI run, and a redeploy. That build-deploy cycle kills the fast try-a-tweak-see-if-it-improves loop that makes AI features good.

Prompts are config, not code. AWS SSM Parameter Store is the baseline place to keep that config: a plain key/value store, free for standard parameters, and reachable from any service with an IAM role.

Read the prompt from SSM at boot

Store the template at a versioned key like /prompts/summarizer/v1, then fetch it once when the module loads. boto3 reads the parameter with a single get_parameter call and pulls the string out of the response payload:

import os
import boto3


def _load_prompt() -> str:
    ssm = boto3.client(
        "ssm",
        endpoint_url=os.environ.get("AWS_ENDPOINT_URL"),
        region_name=os.environ.get("AWS_DEFAULT_REGION", "us-east-1"),
    )
    return ssm.get_parameter(Name="/prompts/summarizer/v1")["Parameter"]["Value"]


PROMPT_TEMPLATE = _load_prompt()


def render_prompt(text: str) -> str:
    return PROMPT_TEMPLATE.format(text=text)

The value comes back nested: get_parameter returns a dict with a Parameter object, and the string you want is ["Parameter"]["Value"]. Loading it at module import time means the round trip to SSM happens once per process, not on every request.

Seed and update the parameter

Write the template with put-parameter. Use --type String for a plain prompt (SecureString if it ever holds a secret):

aws ssm put-parameter \
    --name /prompts/summarizer/v1 \
    --value "Summarize the following in two sentences, formal tone:\n{text}" \
    --type String

Read it back to confirm what the service will load:

aws ssm get-parameter --name /prompts/summarizer/v1

Now iterating on the prompt is a single command. To roll out "two sentences, formal tone," you run put-parameter --overwrite and restart (or re-fetch) the service - no PR, no CI, no image build. Verify end to end by running the service and checking the rendered output:

python3 summarizer.py

Versioning and next steps

The v1 suffix in the key name is a cheap version scheme: publish /prompts/summarizer/v2, point the service at it, and you can roll back by flipping one env var. SSM also keeps a parameter history automatically, so you can see prior values.

This is the baseline. Production prompt-config systems - LangSmith, Helicone, PromptLayer - version prompts, A/B test them, and link each version to eval results. SSM Parameter Store gets you the core win, decoupling prompts from deploys, with tooling you already have in AWS.