AWS-SQS and SNS Summary

Brief Points about SQS:

• SQS is fast reliable fully managed messaging Queue Service
• It is a webservice that gives you access to message queues that stores messages waiting to be processed.
• It offers reliable highly scalable hosted queue for storing messages between servers.
• It allows the decoupling of application components such that a failure in one component does not cause a bigger problem to application functionality
• Using SQS you no longer need a highly available message cluster or the burden of running it.
• You can delete all the messages in SQS queue without deleting the SQS Queue itself.
• You can use applications on EC2 instances to read and process the SQS Queue message.
• You can use Autoscaling to scale the EC2 fleet processing the SQS messages, as the queue size increases.
• These applications on EC2 instances can process the SQS messages/jobs than post the results to other SQS queues or other AWS services.

AWS Queue Types:
Standard Queue
1.High thoughput
2.At least one delivery
3.Duplicacy is possible
4.Best effort ordering

FIFO Queue:

• 1. Limited throughput (300 TPS)
• 2. Exactly once processing.
• 3.Duplicacy not possible
• 4.FIFO ques are limited to transaction per second (TPS)
• 5. It have all the capability of standard queue

SQS Pricing:
The first 1 million monthly requests are free, after that pricing is according to Regions. eg in Mumbai Region–Standard Queue–$0.40/ million requests
FIFO Queue–0.50 /million requests.

HOW Amazon SQS Charges:
1.An action-Every Amazon SQS action count as a request.
2.FIFO Request–API action for sending, receiving, deleting and changing visibility of messages from FIFO queue are charged at FIFO rates.
3.Contents of Requests–A single request can have from 1 to 10 messages, up to a maximum total payload of 256kb
4.Size of Payload-Each 64kb chunk is build as one request.

Amazon SQS Polling: There are two types of polling mechanisms supported by Amazon SQS for consuming messages -short polling and long polling.
Short polling:
1 A request is returned immediately even after queue is empty.
2.It does not wait for messages to appear in the queue.
3.It queries only the subsets of available servers for messages (based on weighted random distribution).
4.Default by SQS. This is default polling mechanism uses for Amazon SQS queue consumers that do not specifically choose polling options.
5.More requests are used which implies higher cost
6.False empty responses–Amazon SQS read requests may return empty responses even if there are messages in the queue.

Long polling:
1.With long polling, the RecieveMessage request queries all of the servers for message. Amazon SQS sends a response after it collects at least one one available message.
2.Amazon SQS sends an empty response only if the polling wait time expires.
3.Long polling may take a little longer to fulfill each request, but is guaranteed to return a message as long as there are messages in the queue.
4.It not only eliminates false empty responses but also reduces number of empty responses buy using connection time out (20 Sec) , thus leading to significant cost benefits.

Long polling is preferable over short polling in most cases.

Short polling is billed the same as long polling.

• SQS Retention period:
• SQS messages can remain in the queue for upto 14 days
• Range is 1 min to 14 days (deafult is 4 days)
• Once the maximum retention period is reached, it will be automatically deleted from the queue.
• Messages can be sent to the queue and read from the queue simultaneously
• SQS can be used with DynamoDB,EC2, ECS,Redshift,RDS, lamda,S3 to make destinated/decoupled applications.
• You can have multiple queues with different priorities.

• SQS Visibility Timeout:
  It is the duration of time a message is locked for read by other servers.
• Maximum is 12 hours and default is 30 Secs and minimum is 0 seconds.
• A server that read a message to process it can change the message visibility timeout if it needs more time to process the message.

• After a message is read, these are following possibilities:
• 1.An Acknowledgement is received that the message is processed , so it must be deleted from queue to avoid duplication
• 2.If a fail is received, or the visibility timeout expires, the message will then be visible for read, such that it can be read and processed by another server.

• Delivery Delay:
• AWS SQS promotes delivery delay options to postpone the delivery of new messages to a queue .
• If delivery delay is defined for a queue, any new messages will not be visible to the server for the duration of delay.The default(min) delay for a queue is 0 seconds.
• The maximum is 15 minutes.

Receive message wait time: The default time is 0 seconds. This is max amount of time that a long polling receive can wait for a message to become available before returning an empty response (Max Value is 20 sec).

Dead Letter queue:

• The main task of a dead letter queue is handling message failure. A dead letter queue lets you set aside and isolate messages that can’t be processed correctly to determine why their processing didn’t succeed.
• Amazon SQS does not create the dead letter queue automatically. You must first create the queue before using it as a dead-letter queue.

• When the recievecount for a message exceeds the maxrecievecount for a queue, Amazon SQS moves the message to a dead-letter queue.
• The dead-letter queue of a FIFO queue must also be a FIFO queue.Similarly the dead letter queue of a standard queue must also be a standard queue.
• Dead letter queue must reside in same region as other queues that use the dead letter queue.

• Dead letter queue allows you to do the following:
• Configure an alarm for any messages delivered to a dead letter queue.
• Examine logs for exception that might have caused messages to be delivered to a dead letter queue.
• Analyze the contents of messages delivered to a dead letter queue to diagonise software or the producers or consumers hardware issues.
• Determine whether you have given your consumer sufficient time to process message.

• When should be dead letter queue used:
• Do use dead letter queue with standard queues- You should always take advantages of dead letter queue when your application don’t depend on ordering.
• Do use dead letter queue to decrease the number of messages and to reduce the possibility of exposing your system to poison pill messages.
• Don’t use dead letter queue with standard queues when you want to be able to keep retrying the transmission of message indefintly.
• Don;t use dead letter queue with FIFO Queue if you don;t want to break the exact order of message or operations.

Viewing messages using the console might cause messages to be moved to dead letter queue-To solve this you can increase the Maximum Receives settings for the corresponding queues redrive policy.

Default Visibility Timeout–Value must be between 0 seconds and 12 hours. It is the duration of time a message is locked for read by other servers.
Message Retention period–Value must be between 1 minute and 14 days. Once the maximum retention period is reached, it will be automatically deleted from the queue.
Maximum Message Size–Value must be between 1 and 256KB.
Delivery Delay–Value must be between 0 seconds and 15 minutes. AWS SQS promotes delivery delay options to postpone the delivery of new messages to a queue
Receive Message wait time: Value must be between 0 and 20 Secs. This is max amount of time that a long polling receive can wait for a message to become available before returning an empty response

Amazon SQS Temporary Queue Client-This client makes it easy to create and delete many temporary messaging destinations without inflating your AWS bill.

The most common use case for temporary queues is the request-response messaging pattern, where a requester creates a temporary queue for receiving each response message. To avoid creating an Amazon SQS queue for each response message, the Temporary Queue Client lets you create and delete multiple temporary queues without making any Amazon SQS API calls.

---

SNS

• SNS is a fast, flexible, fully managed push notification service
• It is a web service that coordinate and manages the delivery or sending of messages to subscribing end points or clients.
• It allows for sending individual messages or Fan out messages to a large number of recipients or to other distributed AWS services.
• Messages published to an SNS topic will be delivered to the subscription immediately.
• Inexpensive, pay as you go model with no upfront cost.
• Reliable–At least three copies of the data are stored across multiple AZ in same region.
• It is a way of sending message when you are using auto scaling, it triggers an SNS service which will email you that your EC2 instance is growing.

Publisher–SNS Topic—Lamda,SQS, HTTPS, Email,SMS

Publishers–Publishers are also known as producers that produce and send the message to the SNS which is a logical access point.
Subscribers–Subscribers such as WebServers, email addresses, Amazon SQS queues, AWS Lamda receives the message or notification from the SNS over one of the supported protocols (Amazon SQS, email, Lamda, HTTPS, SMS).

SNS Topic:
It is logical access point and communication channel.
Each topic has a unique name
A topic name is limited to 256 alphanumeric characters
The Topic name must be unique within the AWS account
A topic can support subscribers’ and notification delivers over multiple protocols.
Messages/request published to a single topic can be delivered over multiple protocols as configured when creating each subscribers.

Delivery formats/transport protocols–SMS,email,Email-JSON for applications,HTTP/HTTPS,Lamda,SQS.

1. When using amazon SQS, you create a topic and control access to it by defining access policies that determine which publishers and subscribers can communicate with the topic.
2. Instead of including a specific destination address in each message, a publisher sends message to topics that they have created or to topics they have permission to publish to.
3. Amazon SNS matches the topic to a list of subscribers who have subscribed to that topic, and delivers the message to each of the subscribers.
4. Each topic has a unique name that identifies the Amazon SNS endpoints for publishers to post messages.
   All subscribers to a topic receive same message.
5. By Default only the topic owner can push the message to the SNS topics. The owner can set/change permissions to one or more users to publish to his topics.
6. Only the owner of the topic can grant/change permissions for the topic.
7. Subscribers can be those with or without AWS id. Only Subscribers with AWS id can request subscriptions to SNS topic.
   When you publish a notification to a topic, SNS will send identical copies of that message to each end point subscribers to that topic.

Amazon SNS alternatives:
Amazon Kenisis Data Stream
Amazon Managed Queue Service
Apache Kafka
Twilio
Pusher

Amazon SNS Pricing:
1.Publish action–Each 64KB of request payload count as one request, so 256KB payload will be charged as 4 request.
2.Mobile Push Notification–For eg $0.50/million request
3.SMS–Price depends on country
4.Email–$2/100000
5.HTTPS/Notification–$0.60/million
6.SQS and Lamda calls are free–These are charged as SQS and Lamda rates.
7.Data Transfer