Azure messaging services — Event Grid, Event Hubs, and Service Bus
Azure offers three services that assist in transfer of events or messages between distributed applications with the goal of creating integrated but loosely coupled system.
· Azure Event Grid
· Azure Event Hubs
· Azure Service Bus
Messaging services at high level solves following design problems:
1. Decoupling: Applications are decoupled into smaller, independent building blocks that are easier to develop, deploy and maintain. Messaging service provide communication and coordination for these distributed applications.
2. Performance: Messaging service enables asynchronous communication. Producers can add requests to the queue without waiting for them to be processed. Consumers process messages only when they are available. No component in the system is ever stalled waiting for another, optimizing data flow
3. Scalability: When workloads peak, multiple instances of your application can all add requests to the queue without risk of collision. As your queues get longer with these incoming requests, you can distribute the workload across a fleet of consumers. Producers, consumers and the queue itself can all grow and shrink on demand.
Before we look at each of these services lets take a high level look at What is a queue, message and event
How Queue works
Queues have two ends. The producer end is where messages are put into the queue. The other end is the consumer end, where a consumer pulls messages off the top of the queue. A queue is a FIFO structure: first message in, first message out.
What is a message?
Message is the raw data which producer sends to the queue and later on receiver reads from the queue. Producer of message puts the message on the queue with the expectation about how consumer will handle a message.
What is an Event?
An event is a notification of a condition or a state change. The producer of the event has no expectation about how the event is handled. The consumer of the event decides what to do with the notification. Event does not contain actual data object, it only contains state change notification based on which consumer will react.
Azure Event Grid
Azure Event Grid is a managed event routing service which routes events from Azure and non-Azure resources. It distributes the events to registered subscriber endpoints. Event Grid distributes events from different sources to different handlers, such as Azure Functions or logic apps.
Azure services have predefined topics to which messages could be published. These messages are available for subscription from the appropriate source. For custom topics, you simply create your Event Grid topic and then begin publishing and setting up subscriptions. The topic has an endpoint to publish messages and a set of keys for secure access.
Use case for Event Grid
Use event grid when you want to implement PUSH model i.e. you want to avoid long polling. Instead, events are routed immediately to an endpoint that you specify. In many cases it can also lead to cost savings because it removes the overhead of polling on a regular basis and instead triggers code only when it is needed to consume an event. For example you want to take action when a new user is added in resource group or a file is uploaded to blob storage.
Azure Event Hub
Event Hub is big data stream messaging service i.e. assume that data is continuously sent to this service. It can receive and process millions of messages per second. Azure Event Hubs supports the Kafka and AMQP 1.0 messaging protocols. Azure Event Hubs is recommended when working with a real-time event stream. Existing Kafka clients can be connected to an Azure Event Hub, providing an alternative to deploying your own Kafka cluster. Event Hubs is a fully managed Platform-as-a-Service (PaaS) with little configuration or management overhead, so you focus on your business solutions.
In Event hubs the data in the partition stays there even after it’s been read by the consumer. This allows the consumers to come and read the data again if required (example lost connection). The events stored in partitions get deleted only after the retention period is expired (example 1 day). You cannot manually delete data from partitions.
Use case for Event Hub
As we discussed above, use Event hub when we want to process millions of messages per second. Real-world use cases include getting telemetry data from cars, games, IoT scenarios where millions of devices push data to the cloud, gaming scenarios where you push user activities at scale to the cloud, etc.
Azure Service Bus
Azure Service Bus is a fully managed, secure and reliable Platform-as-a-service (PaaS) message broker for asynchronous transfer of data between applications. Azure Service Bus support both Queues and Topic/Subscriptions.
Service Bus is a brokered messaging system. It stores messages in a “broker” (for example, a queue) until the consumer is ready to receive the messages. Publisher sends message to queue, queue replicates message and send acknowledge back to publisher. Receiver reads the message and sends acknowledgement back to queue for deletion of message (Queue moves message to Dead letter queue on deletion request).
Use Case for Service Bus
It is useful when an application wants to send a message to other applications and expect them to perform a specific action eg processing order or performing series of workflow after a user is added to a resource group.
Summary
In this article we had a quick introduction to Azure messaging services. The decision to choose between them should be dependent on the kind of data which needs to be sent and processed between Sender and Receiver
Reference
1. Azure Service Bus: https://azure.microsoft.com/en-in/services/service-bus/
2. Azure Event Hub: https://docs.microsoft.com/en-us/azure/event-hubs/event-hubs-about
3. Azure Event Grid: https://docs.microsoft.com/en-us/azure/event-grid/overview
