How to maintain SseEmitters list between multiple instances of a microservice?

In a microservice architecture, Server-Sent Events (SSE) is a technology that enables servers to push real-time data to clients. SseEmitter is a mechanism for implementing SSE in the Spring framework. When used in a multi-instance microservice environment, maintaining a consistent list of SseEmitter across instances can present challenges. Below are some strategies for maintaining a list of SseEmitter across multiple instances in microservices:

1. Central Storage

Central storage, such as Redis or other distributed caching/databases, can be used to store information about all active SseEmitter instances. Each microservice instance can read and update this information from the central storage. However, SseEmitter itself cannot be serialized, so we store the session or user identifiers along with their corresponding instance information.

Example:

• When a user connects, the microservice instance creates a new SseEmitter and stores the session ID and the current instance identifier in the central storage.
• When events need to be sent, all instances check the central storage, and only the instance with the corresponding session ID sends the event to the client.
• When SseEmitter times out or disconnects, the relevant instance is responsible for removing the corresponding session ID from the central storage.

2. Message Queues and Event Buses

Using message queues (such as RabbitMQ, Kafka, etc.) or event buses (such as Spring Cloud Stream) to publish events, all instances can subscribe to these events and send data only to clients connected via that instance.

Example:

• When data needs to be broadcast, the service instance publishes the event to the message queue or event bus.
• All microservice instances subscribe to these events and check if they have an SseEmitter associated with the user.
• If so, the corresponding instance sends the information to the client via SseEmitter.

3. Sticky Sessions in Load Balancers

Configure the load balancer (such as Nginx or AWS ELB) to use sticky sessions, ensuring that all requests from a specific client are routed to the same service instance. This enables each instance to manage SseEmitter independently, as all related requests are routed to the instance that created the corresponding SseEmitter.

Example:

• When a client's first request is routed to instance A, instance A creates an SseEmitter and manages it.
• Due to sticky session configuration, subsequent requests are routed to instance A, so only instance A needs to maintain the SseEmitter.

Considerations

• Fault Tolerance: If an instance fails, a mechanism should be in place to reroute connections to other instances, and it may be necessary to recreate SseEmitter.
• Data Consistency: If there is state or information that needs to be shared across instances, ensure data consistency.
• Performance: Using central storage or message queues may increase latency; performance testing is required to ensure the system's response time is acceptable.
• Security: When using these methods, ensure all communications are encrypted and access permissions are appropriately managed.

Depending on the specific circumstances and requirements of the microservice, choose the most suitable method or combine several methods to achieve a more robust and resilient solution.