Queue Manager Swiftlet (HA)

Overview

The HA Queue Manager Swiftlet extends the standard CE Queue Manager by adding high availability (HA) features for active/standby failover. It ensures that queue state, duplicate detection logs, and message group assignments are synchronously replicated to a standby node, enabling seamless failover with no message loss or duplication. For standard queue management features, refer to the CE Queue Manager documentation.

Features

Synchronous Queue Replication for HA

In HA mode, all queue state changes—including message enqueues, dequeues, and duplicate detection logs—are synchronously replicated from the active node to the standby node using a dedicated replication tunnel. This ensures that the standby node maintains an up-to-date copy of all queue data and can take over immediately in the event of a failover. The replication protocol covers persistent and non-persistent messages, message group assignments, and duplicate detection backlogs. Replication is performed before acknowledging message operations to clients, guaranteeing strong consistency between active and standby nodes.

Replication Tunnel Management

The Swiftlet establishes a dedicated replication tunnel named queue between the active and standby nodes. The tunnel uses a specific protocol version (currently 600) to exchange queue state, duplicate logs, and message group information. If the tunnel is unavailable, replication is disabled and a warning is logged.

Failover and State Transition Handling

The Swiftlet manages transitions between active and standby states. On becoming active, it initializes the replication source and synchronizes all queues and message groups. On standby, it listens for replication items and applies them to local queue state. During failover, the standby node takes over as active, using the replicated state to continue processing without message loss or duplication.

Duplicate Detection with Replication

Duplicate message detection is always enabled by default in HA mode and is tightly integrated with replication. The duplicate detection backlog for each queue is maintained and replicated to the standby node. Any changes to the backlog—such as insertion or removal of message IDs—are sent as replication items and applied on the standby. This ensures that after failover, the new active node continues duplicate detection seamlessly, preventing message duplication.

Backlog Size and Synchronization

The size of the duplicate detection backlog is configurable per queue. Any changes to the backlog size or enabled/disabled state are replicated to the standby. The standby node reconstructs the backlog using replicated events and bulk requests, ensuring consistency.

Transactional Replication

Duplicate detection changes are batched and sent as bulk requests before transaction completion, ensuring atomicity and consistency across active and standby nodes.

Configuration Example:

<swiftlet name="sys$queuemanager">
  <queues>
    <queue name="orders" duplicate-detection-enabled="true" duplicate-detection-backlog-size="200"/>
  </queues>
</swiftlet>

Replicated Message Group State

For clustered queues with message group support enabled, the assignment of message groups to consumers is replicated to the standby node. All group creation, update, and removal events are sent as replication items. This ensures that after failover, the standby node preserves message group semantics and continues dispatching messages according to the established group assignments.

Group State Initialization

On activation, the full set of message group assignments is sent to the standby node using initialization requests. Subsequent changes are replicated incrementally.

Configuration Example:

<swiftlet name="sys$queuemanager">
  <clustered-queues>
    <clustered-queue name="myclusteredqueue" message-group-enabled="true"/>
  </clustered-queues>
</swiftlet>

Configuration Guide

Increasing Duplicate Detection Backlog Size for a Queue in HA

Use this scenario when you expect a high rate of duplicate messages or require a longer history for duplicate detection on a specific queue in an HA environment. Increasing the backlog size ensures that more message IDs are tracked and replicated to the standby node.

Edit the routerconfig.xml on both active and standby nodes.
Locate the desired queue under the queues entity list.
Set the duplicate-detection-backlog-size attribute to the desired value.
Restart the affected queue or the router for the change to take effect.

<swiftlet name="sys$queuemanager">
  <queues>
    <queue name="orders" duplicate-detection-backlog-size="500"/>
  </queues>
</swiftlet>

Configuration Reference

The top-level entity in routerconfig.xml is <swiftlet name="sys$queuemanager">.

`<swiftlet name="sys$queuemanager">` Properties

These properties are attributes of the <swiftlet name="sys$queuemanager"> entity.

Parameter	Type	Default	Mandatory	Reboot Required	Description
`collect-interval`	Long	`1000`	No	No	Interval for collecting the Number of Messages per Queue
`cluster-metric-interval`	Long	`10000`	No	No	Interval for sending metrics about clustered queues (min: 1000)
`max-flowcontrol-delay`	Long	`5000`	No	Yes	Maximum Flow Control Delay
`log-expired-messages`	Boolean	`false`	No	No	Log Expired Messages to the 'Warning' Log File
`log-duplicate-messages`	Boolean	`false`	No	No	Log Duplicate Messages to the 'Warning' Log File
`deliver-expired-messages`	Boolean	`false`	No	No	Deliver Expired Messages to Clients
`cluster-topic`	String	`swiftmq.cluster`	No	Yes	Cluster Message Exchange Topic
`multi-queue-transaction-global-lock`	Boolean	`false`	No	No	Use a global lock for multi Queue Transactions

<swiftlet name="sys$queuemanager" collect-interval="1000" cluster-metric-interval="10000" max-flowcontrol-delay="5000" log-expired-messages="false" log-duplicate-messages="false" deliver-expired-messages="false" cluster-topic="swiftmq.cluster" multi-queue-transaction-global-lock="false"/>

`<queue-controllers>` in `<swiftlet name="sys$queuemanager">`

Controllers for System Queues

Each <queue-controller> entry is identified by its name attribute (the System Queue Controller).

Parameter	Type	Default	Mandatory	Reboot Required	Description
`predicate`	String	—	Yes	No	Queue Name SQL-Like Predicate
`cache-size`	Integer	`500`	No	No	Message Queue Cache Size
`cache-size-bytes-kb`	Integer	`-1`	No	No	Message Queue Cache Size Bytes (KB)
`cleanup-interval`	Long	`120000`	No	No	Interval for checking of msg expiration
`flowcontrol-start-queuesize`	Integer	`400`	No	No	Flow Control starts at this Queue Size
`max-messages`	Integer	`-1`	No	No	Max. Messages for this Queue
`persistence-mode`	String	`as_message`	No	No	Persistence Mode of this Queue (choices: `as_message`, `persistent`, `non_persistent`)
`consumer-mode`	String	`shared`	No	No	Consumer Mode (choices: `shared`, `exclusive`, `activestandby`)
`duplicate-detection-enabled`	Boolean	`true`	No	No	Enables/Diables Duplicate Message Detection
`duplicate-detection-backlog-size`	Integer	`100`	No	No	Size of the Backlog for Duplicate Message Detection
`monitor-alert-threshold`	Integer	`-1`	No	No	Threshold in Number of Messages after which the Monitor Swiftlet sends a Queue Monitor Alert

<swiftlet name="sys$queuemanager">
  <queue-controllers>
    <queue-controller name="..." predicate="..."/>
  </queue-controllers>
</swiftlet>

`<composite-queues>` in `<swiftlet name="sys$queuemanager">`

Composite Queues

Each <composite-queue> entry is identified by its name attribute (the Composite Queue).

<swiftlet name="sys$queuemanager">
  <composite-queues>
    <composite-queue name="..."/>
  </composite-queues>
</swiftlet>

`<queue-bindings>` in `<composite-queues>`

Queue Bindings

Each <queue-binding> entry is identified by its name attribute (the Queue Binding).

Parameter	Type	Default	Mandatory	Reboot Required	Description
`message-selector`	String	—	No	No	Optional Selector to filter messages for this Binding
`generate-new-message-id`	Boolean	`false`	No	No	Generates a new Message Id for this Binding's Message Copies
`change-destination`	Boolean	`false`	No	No	Changes the JMS Destination of Message Copies to the Bindung Queue
`default-delivery`	Boolean	`false`	No	No	A defaut delivery gets the message if no other binding matches.

<swiftlet name="sys$queuemanager">
  <composite-queues>
    <composite-queue name="...">
      <queue-bindings>
        <queue-binding name="..."/>
      </queue-bindings>
    </composite-queue>
  </composite-queues>
</swiftlet>

`<topic-bindings>` in `<composite-queues>`

Topic Bindings

Each <topic-binding> entry is identified by its name attribute (the Topic Binding).

Parameter	Type	Default	Mandatory	Reboot Required	Description
`message-selector`	String	—	No	No	Optional Selector to filter messages for this Binding
`generate-new-message-id`	Boolean	`false`	No	No	Generates a new Message Id for this Binding's Message Copies
`default-delivery`	Boolean	`false`	No	No	A defaut delivery gets the message if no other binding matches.

<swiftlet name="sys$queuemanager">
  <composite-queues>
    <composite-queue name="...">
      <topic-bindings>
        <topic-binding name="..."/>
      </topic-bindings>
    </composite-queue>
  </composite-queues>
</swiftlet>

`<clustered-queues>` in `<swiftlet name="sys$queuemanager">`

Clustered Queues

Each <clustered-queue> entry is identified by its name attribute (the Clustered Queue).

Parameter	Type	Default	Mandatory	Reboot Required	Description
`message-group-enabled`	Boolean	`false`	No	No	Enables/Diables grouping of Messages
`message-group-property`	String	`JMSXGroupID`	No	No	Name of Property to group Messages
`message-group-expiration`	Long	`-1`	No	No	Time (ms) after the last occurance after which a message group expires
`message-group-expiration-cleanup-interval`	Long	`-1`	No	No	Interval (ms) in which message groups are checked for expiration

<swiftlet name="sys$queuemanager">
  <clustered-queues>
    <clustered-queue name="..."/>
  </clustered-queues>
</swiftlet>

`<queue-bindings>` in `<clustered-queues>`

Queue Bindings

Each <queue-binding> entry is identified by its name attribute (the Queue Binding).

Parameter	Type	Default	Mandatory	Reboot Required	Description
`redispatch-enabled`	Boolean	`true`	No	No	Enables/Diables Redispatch of Messages if no Consumer is attached

<swiftlet name="sys$queuemanager">
  <clustered-queues>
    <clustered-queue name="...">
      <queue-bindings>
        <queue-binding name="..."/>
      </queue-bindings>
    </clustered-queue>
  </clustered-queues>
</swiftlet>

`<queues>` in `<swiftlet name="sys$queuemanager">`

Queue Definitions

Each <queue> entry is identified by its name attribute (the Queue).

Parameter	Type	Default	Mandatory	Reboot Required	Description
`cache-size`	Integer	`500`	No	No	Message Queue Cache Size
`cache-size-bytes-kb`	Integer	`-1`	No	No	Message Queue Cache Size Bytes (KB)
`cleanup-interval`	Long	`120000`	No	No	Interval for checking of msg expiration
`max-messages`	Integer	`-1`	No	No	Max. Messages for this Queue
`persistence-mode`	String	`as_message`	No	No	Persistence Mode of this Queue (choices: `as_message`, `persistent`, `non_persistent`)
`consumer-mode`	String	`shared`	No	No	Consumer Mode (choices: `shared`, `exclusive`, `activestandby`)
`flowcontrol-start-queuesize`	Integer	`400`	No	No	Flow Control starts at this Queue Size
`duplicate-detection-enabled`	Boolean	`true`	No	No	Enables/Diables Duplicate Message Detection
`duplicate-detection-backlog-size`	Integer	`100`	No	No	Size of the Backlog for Duplicate Message Detection
`monitor-alert-threshold`	Integer	`-1`	No	No	Threshold in Number of Messages after which the Monitor Swiftlet sends a Queue Monitor Alert

<swiftlet name="sys$queuemanager">
  <queues>
    <queue name="..."/>
  </queues>
</swiftlet>

Queue Manager Swiftlet (CE)