skills.rest
richardnroman
richardnromanCOMMUNITY
Explore all skills by this author →
GitHub

aws-messaging-and-streaming

Guides selection and configuration of AWS messaging and streaming services.

Updated Jul 21, 2026
Search Tags:#aws#kafka#eventbridge#sqs#sns#kinesis#event-driven architecture
One-click install
npx skills add https://github.com/richardnroman/Synthetic-Data-Exchange-Licensing-Platform --skill aws-messaging-and-streaming-richardnroman
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: aws-messaging-and-streaming
Source: https://github.com/richardnroman/Synthetic-Data-Exchange-Licensing-Platform/tree/main/.agents/skills/aws-messaging-and-streaming
Command: npx skills add https://github.com/richardnroman/Synthetic-Data-Exchange-Licensing-Platform --skill aws-messaging-and-streaming-richardnroman

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Choosing between AWS messaging and streaming services is error-prone, and misconfigured queues, DLQs, and credentials cause silent message loss or security gaps. This Skill provides decision guidance and integration gotchas for SQS, SNS, EventBridge, Amazon MQ, Kinesis Data Streams, Data Firehose, Managed Service for Apache Flink, and Amazon MSK. ## Core Features & Use Cases - Pattern Selection: Distinguishes messaging (decoupled, consume-once) from streaming (durable, replayable) workloads and maps each to the right AWS service. - Service Comparison: Compares SQS, SNS, EventBridge, Amazon MQ, Kinesis Data Streams, Firehose, Flink, and MSK by differentiators like ordering, retention, and scaling units. - Integration Gotchas: Documents real pitfalls such as SQS system vs. user attributes, SNS-to-Firehose newline handling, DLQ queue policies, and Secrets Manager requirements for MSK SASL/SCRAM. - Use Case: When designing an event-driven architecture, ask which service fits a fan-out notification workload and get a recommendation for SNS with SQS subscriptions, plus the correct DLQ policy with aws:SourceArn conditions. ## Quick Start Ask which AWS service fits your workload, for example whether to use SQS or Kinesis for decoupling microservices with replay requirements.

Frequently Asked Questions about aws-messaging-and-streaming

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I choose between SQS and Kinesis Data Streams?▼

Choose SQS for task queues and decoupled work distribution where messages are consumed once and deleted. Choose Kinesis Data Streams when you need replayable, ordered records with multiple independent consumers reading the same data at different positions.

What is the difference between messaging and streaming on AWS?▼

Messaging deletes messages after consumption and suits command and task distribution workloads. Streaming retains records for a configurable window with strict per-partition ordering, supporting replay, event sourcing, and real-time analytics.

Why is my EventBridge DLQ silently dropping messages?▼

Attaching a DLQ alone is not enough; the DLQ needs a queue policy allowing sqs:SendMessage for the events.amazonaws.com service principal scoped with aws:SourceArn to the rule ARN. The same applies to SNS subscription DLQs with the sns.amazonaws.com principal.

Does Amazon MSK require Secrets Manager for SASL/SCRAM credentials?▼

Yes, MSK SASL/SCRAM secrets must use the AmazonMSK_ prefix and be encrypted with a customer-managed KMS key, then attached via BatchAssociateScramSecret. Secrets using the default aws/secretsmanager key cannot be associated with a cluster.

When should I use Amazon MQ instead of SQS or SNS?▼

Use Amazon MQ when migrating existing applications that rely on JMS, AMQP, or MQTT protocols via ActiveMQ or RabbitMQ. For new cloud-native workloads, SQS and SNS are fully managed and avoid broker administration.

Related Skills

Explore More Software Engineering Skills→
roanbrasilroanbrasil
5

messaging-brokers

Select messaging brokers and implement reliability patterns for distributed systems.

Community
Advanced
awsaws
2.3k

connecting-lambda-to-dynamodb

Automate Lambda-DynamoDB integration with IAM roles and event source mappings.

Official
Advanced
awsaws
2.5k

aws-messaging-and-streaming

Guides selection and use of AWS messaging and streaming services.

Official
Intermediate
armoin2018armoin2018
4

ailey-data-kafka

Automate Kafka workflows for real-time data pipelines across Confluent Cloud, Platform, Apache Kafka, and AWS MSK.

Community
Advanced