AWS SNS + SQS Fanout Pattern
The SNS + SQS fanout pattern lets one event reach many consumers without the producer knowing about any of them. The producer publishes to an SNS topic; each consumer has its own SQS queue subscribed to it. Here's how and why.
The problem: a producer wired to every consumer
When the order service POSTs directly to notifications and analytics, adding a third consumer means editing and redeploying the producer. Producers and consumers are coupled - every new subscriber is a change to the source of truth.
The fix: publish once to SNS, fan out to SQS
- The producer publishes one message to an SNS topic - and is done. It doesn't know or care who's listening.
- Each consumer owns an SQS queue subscribed to the topic. SNS delivers a copy of every message to every subscribed queue.
- Adding a consumer = create a queue + subscribe it. Zero producer changes.
import json, boto3
sns = boto3.client("sns")
ORDER_TOPIC_ARN = "arn:aws:sns:...:order-events"
def publish_order_event(order: dict) -> None:
sns.publish(TopicArn=ORDER_TOPIC_ARN, Message=json.dumps(order))
That's the whole producer. Each consumer polls its own queue independently.
Why SNS and SQS (not just one)
- SNS alone is push + fire-and-forget - if a consumer is down, it misses the message.
- SQS alone is one queue - multiple consumers would compete for the same messages, not each get a copy.
- SNS -> SQS gives you both: a copy per consumer (fanout) and durable buffering per consumer (each queue holds messages until that consumer processes them, with retries + a DLQ).
Setting it up
- Create the SNS topic.
- Create one SQS queue per consumer.
- Subscribe each queue to the topic, and grant the topic permission to send to the queue.
- (Recommended) enable raw message delivery so consumers get the bare message, not the SNS envelope.
Now consumers are fully decoupled: each scales, retries, and fails independently, and the producer stays untouched as the system grows.
Want to try it hands-on? HeyDevJob gives you this exact setup in a live cloud workspace in your browser - edit it, run it, and see it work. Free, nothing to install.
Try it in a workspace →What you'll practice
- Publishing events to an SNS topic from the producer
- Subscribing per-consumer SQS queues for fanout
- Reasoning about why SNS+SQS beats either alone
FAQ
What is the SNS to SQS fanout pattern?
The producer publishes one message to an SNS topic; each consumer has its own SQS queue subscribed to that topic, so every consumer gets a copy. Adding a consumer means adding a queue + subscription - no producer change.
Why use SNS and SQS together instead of just one?
SNS alone is fire-and-forget (a down consumer misses messages); SQS alone makes consumers compete for the same messages. SNS->SQS gives each consumer its own durable, buffered copy with independent retries and DLQs.
How do I add a new consumer to an SNS fanout?
Create a new SQS queue, subscribe it to the SNS topic, and grant the topic permission to send to it. The producer keeps publishing to the same topic, unchanged.
Keep learning
Learn it by doing. Open this in a live cloud workspace, make the change yourself, and keep a record of the work you can share.
Open the workspace →