Queue Manager Swiftlet (HA)

Overview

The HA (High Availability) Queue Manager Swiftlet extends the standard CE Queue Manager with active/standby failover capabilities. It ensures queue state replication, duplicate message detection, and seamless failover between active and standby routers. For standard queue management features such as queue creation, cache sizing, persistence, and composite/clustered queues, refer to the CE Queue Manager documentation.

Features

Active/Standby Replication of Queue State

In HA mode, all queue state—including messages, duplicate detection logs, and message group assignments—is synchronously replicated from the active router to the standby router. This is achieved via a dedicated replication tunnel (queue) and a protocol that transmits all relevant queue operations. Upon failover, the standby router takes over with a consistent and up-to-date view of all queues, ensuring no message loss or duplication.

Replication is protocol-versioned (currently version 600) and includes the full initialization of queue state (including duplicate detection backlogs and message group assignments) when a queue is started or when the active/standby roles change. The replication protocol is robust against network interruptions and ensures transactional consistency by batching changes and applying them atomically on the standby.

Replication Source and Sink

The active router acts as a replication source, publishing queue state changes to the standby. The standby acts as a replication sink, applying incoming changes to its local queue structures. Specialized proxy classes (HAMessageQueueSourceProxyImpl, HAMessageGroupDispatchPolicySourceProxyImpl) handle the marshalling and transmission of queue and group state.

Synchronous Replication Before Acknowledgment

Queue operations that affect replicated state (such as message enqueue, dequeue, duplicate log updates, and message group changes) are not acknowledged to clients until the corresponding replication protocol request has been sent to the standby. This ensures that, in the event of a failover, the standby router can resume operation without message loss or duplicate delivery.

Duplicate Message Detection in HA

Duplicate detection is enabled by default on all queues in HA mode and is fully replicated between active and standby routers. The duplicate detection backlog is initialized and maintained in sync, and all changes (insertions and removals of message IDs) are transmitted as part of the replication protocol. During failover, the standby router reconstructs the duplicate backlog to ensure that no duplicate messages are delivered to consumers.

The backlog size and enablement can be configured per queue, and any changes are propagated to the standby. The protocol ensures that only persistent messages are tracked for duplicate detection, and temporary queues are excluded from replication.

Backlog Synchronization and Recovery

When a queue is started on the standby (either at initial sync or after failover), the duplicate backlog is rebuilt from the replicated state. If there are in-doubt requests (e.g., due to a failover during a transaction), the protocol ensures that all relevant duplicate IDs are correctly applied or discarded, maintaining consistency.

Configuration Example:

<swiftlet name="sys$queuemanager">
  <queues>
    <queue name="orders" duplicate-detection-enabled="true" duplicate-detection-backlog-size="200"/>
  </queues>
</swiftlet>

HA Message Group Replication

Message group assignments for clustered queues are also replicated between the active and standby routers. All group creation, dispatch, and removal operations are transmitted via the replication protocol. This ensures that, after failover, message group affinity is preserved and consumers continue to receive the correct message groups without interruption or reassignment.

Group State Initialization

When a clustered queue with message grouping is started, the full set of group assignments is transmitted to the standby as part of the replication protocol. Any subsequent changes (such as group expiration or reassignment) are also replicated in real time.

Configuration Example:

<swiftlet name="sys$queuemanager">
  <clustered-queues>
    <clustered-queue name="myclusteredqueue" message-group-enabled="true"/>
  </clustered-queues>
</swiftlet>

HA-Specific Configuration Parameters

The HA Queue Manager Swiftlet exposes several configuration parameters relevant for HA operation. These include duplicate detection settings (enabled by default in HA), backlog size, and all standard queue and clustered queue properties. While most parameters are inherited from the CE edition, in HA mode, the settings for duplicate detection and message group replication are always enforced and replicated.

Duplicate Detection Defaults

In HA mode, duplicate-detection-enabled is true by default for all queues, and the default duplicate-detection-backlog-size is 100. These can be overridden per queue as needed.

Configuration Example:

<swiftlet name="sys$queuemanager">
  <queues>
    <queue name="payments" duplicate-detection-enabled="false"/>
  </queues>
</swiftlet>

Failover and State Transition Handling

The HA Queue Manager Swiftlet manages transitions between active and standby states, ensuring that queue state, duplicate logs, and message group assignments are correctly initialized and cleaned up. During failover, the standby router applies all pending replication requests and assumes the active role without message loss. The Swiftlet ensures that all necessary listeners and proxies are registered and deregistered as the router transitions between states.

State Listeners and Adapters

Specialized state adapters (ActiveHAAdapter, StandbyHAAdapter) are registered with the HA Controller Swiftlet to manage the lifecycle of replication sources and sinks, and to coordinate queue state transitions during startup, failover, and shutdown.

Internal Queue Naming

• tpc$<roottopic> — Virtual queue for topic broker invocation

Configuration Guide

Increase Duplicate Detection Backlog for a Queue in HA

If a specific queue is expected to handle a high volume of messages with potential for duplicate delivery (e.g., due to client retries), increase the duplicate detection backlog size to ensure all duplicates are detected and suppressed, even during failover.

1. Edit the routerconfig.xml to locate the sys$queuemanager swiftlet.
2. Within the <queues> section, add or modify the target queue with the desired duplicate-detection-backlog-size.
3. Restart the router for the change to take effect.

<swiftlet name="sys$queuemanager">
  <queues>
    <queue name="invoices" duplicate-detection-backlog-size="500"/>
  </queues>
</swiftlet>

Disable Duplicate Detection for a Specific Queue

In rare cases where duplicate message delivery is acceptable or managed at the application level, you may wish to disable duplicate detection for a specific queue, even in HA mode.

1. Edit the routerconfig.xml and locate the sys$queuemanager swiftlet.
2. Within the <queues> section, set duplicate-detection-enabled="false" for the desired queue.
3. Restart the router for the change to take effect.

<swiftlet name="sys$queuemanager">
  <queues>
    <queue name="auditlog" duplicate-detection-enabled="false"/>
  </queues>
</swiftlet>

Configuration Reference

The top-level entity in routerconfig.xml is <swiftlet name="sys$queuemanager">.

<swiftlet name="sys$queuemanager"> Properties

These properties are attributes of the <swiftlet name="sys$queuemanager"> entity.

Parameter	Type	Default	Mandatory	Reboot Required	Description
collect-interval	Long	1000	No	No	Interval for collecting the Number of Messages per Queue
cluster-metric-interval	Long	10000	No	No	Interval for sending metrics about clustered queues (min: 1000)
max-flowcontrol-delay	Long	5000	No	Yes	Maximum Flow Control Delay
log-expired-messages	Boolean	false	No	No	Log Expired Messages to the 'Warning' Log File
log-duplicate-messages	Boolean	false	No	No	Log Duplicate Messages to the 'Warning' Log File
deliver-expired-messages	Boolean	false	No	No	Deliver Expired Messages to Clients
cluster-topic	String	swiftmq.cluster	No	Yes	Cluster Message Exchange Topic
multi-queue-transaction-global-lock	Boolean	false	No	No	Use a global lock for multi Queue Transactions

<swiftlet name="sys$queuemanager" collect-interval="1000" cluster-metric-interval="10000" max-flowcontrol-delay="5000" log-expired-messages="false" log-duplicate-messages="false" deliver-expired-messages="false" cluster-topic="swiftmq.cluster" multi-queue-transaction-global-lock="false"/>

<queue-controllers> in <swiftlet name="sys$queuemanager">

Controllers for System Queues

Each <queue-controller> entry is identified by its name attribute (the System Queue Controller).

Parameter	Type	Default	Mandatory	Reboot Required	Description
predicate	String	—	Yes	No	Queue Name SQL-Like Predicate
cache-size	Integer	500	No	No	Message Queue Cache Size
cache-size-bytes-kb	Integer	-1	No	No	Message Queue Cache Size Bytes (KB)
cleanup-interval	Long	120000	No	No	Interval for checking of msg expiration
flowcontrol-start-queuesize	Integer	400	No	No	Flow Control starts at this Queue Size
max-messages	Integer	-1	No	No	Max. Messages for this Queue
persistence-mode	String	as_message	No	No	Persistence Mode of this Queue (choices: as_message, persistent, non_persistent)
consumer-mode	String	shared	No	No	Consumer Mode (choices: shared, exclusive, activestandby)
duplicate-detection-enabled	Boolean	true	No	No	Enables/Diables Duplicate Message Detection
duplicate-detection-backlog-size	Integer	100	No	No	Size of the Backlog for Duplicate Message Detection
monitor-alert-threshold	Integer	-1	No	No	Threshold in Number of Messages after which the Monitor Swiftlet sends a Queue Monitor Alert

<swiftlet name="sys$queuemanager">
  <queue-controllers>
    <queue-controller name="..." predicate="..."/>
  </queue-controllers>
</swiftlet>

<composite-queues> in <swiftlet name="sys$queuemanager">

Composite Queues

Each <composite-queue> entry is identified by its name attribute (the Composite Queue).

<swiftlet name="sys$queuemanager">
  <composite-queues>
    <composite-queue name="..."/>
  </composite-queues>
</swiftlet>

<queue-bindings> in <composite-queues>

Queue Bindings

Each <queue-binding> entry is identified by its name attribute (the Queue Binding).

Parameter	Type	Default	Mandatory	Reboot Required	Description
message-selector	String	—	No	No	Optional Selector to filter messages for this Binding
generate-new-message-id	Boolean	false	No	No	Generates a new Message Id for this Binding's Message Copies
change-destination	Boolean	false	No	No	Changes the JMS Destination of Message Copies to the Bindung Queue
default-delivery	Boolean	false	No	No	A defaut delivery gets the message if no other binding matches.

<swiftlet name="sys$queuemanager">
  <composite-queues>
    <composite-queue name="...">
      <queue-bindings>
        <queue-binding name="..."/>
      </queue-bindings>
    </composite-queue>
  </composite-queues>
</swiftlet>

<topic-bindings> in <composite-queues>

Topic Bindings

Each <topic-binding> entry is identified by its name attribute (the Topic Binding).

Parameter	Type	Default	Mandatory	Reboot Required	Description
message-selector	String	—	No	No	Optional Selector to filter messages for this Binding
generate-new-message-id	Boolean	false	No	No	Generates a new Message Id for this Binding's Message Copies
default-delivery	Boolean	false	No	No	A defaut delivery gets the message if no other binding matches.

<swiftlet name="sys$queuemanager">
  <composite-queues>
    <composite-queue name="...">
      <topic-bindings>
        <topic-binding name="..."/>
      </topic-bindings>
    </composite-queue>
  </composite-queues>
</swiftlet>

<clustered-queues> in <swiftlet name="sys$queuemanager">

Clustered Queues

Each <clustered-queue> entry is identified by its name attribute (the Clustered Queue).

Parameter	Type	Default	Mandatory	Reboot Required	Description
message-group-enabled	Boolean	false	No	No	Enables/Diables grouping of Messages
message-group-property	String	JMSXGroupID	No	No	Name of Property to group Messages
message-group-expiration	Long	-1	No	No	Time (ms) after the last occurance after which a message group expires
message-group-expiration-cleanup-interval	Long	-1	No	No	Interval (ms) in which message groups are checked for expiration

<swiftlet name="sys$queuemanager">
  <clustered-queues>
    <clustered-queue name="..."/>
  </clustered-queues>
</swiftlet>

<queue-bindings> in <clustered-queues>

Queue Bindings

Each <queue-binding> entry is identified by its name attribute (the Queue Binding).

Parameter	Type	Default	Mandatory	Reboot Required	Description
redispatch-enabled	Boolean	true	No	No	Enables/Diables Redispatch of Messages if no Consumer is attached

<swiftlet name="sys$queuemanager">
  <clustered-queues>
    <clustered-queue name="...">
      <queue-bindings>
        <queue-binding name="..."/>
      </queue-bindings>
    </clustered-queue>
  </clustered-queues>
</swiftlet>

<queues> in <swiftlet name="sys$queuemanager">

Queue Definitions

Each <queue> entry is identified by its name attribute (the Queue).

Parameter	Type	Default	Mandatory	Reboot Required	Description
cache-size	Integer	500	No	No	Message Queue Cache Size
cache-size-bytes-kb	Integer	-1	No	No	Message Queue Cache Size Bytes (KB)
cleanup-interval	Long	120000	No	No	Interval for checking of msg expiration
max-messages	Integer	-1	No	No	Max. Messages for this Queue
persistence-mode	String	as_message	No	No	Persistence Mode of this Queue (choices: as_message, persistent, non_persistent)
consumer-mode	String	shared	No	No	Consumer Mode (choices: shared, exclusive, activestandby)
flowcontrol-start-queuesize	Integer	400	No	No	Flow Control starts at this Queue Size
duplicate-detection-enabled	Boolean	true	No	No	Enables/Diables Duplicate Message Detection
duplicate-detection-backlog-size	Integer	100	No	No	Size of the Backlog for Duplicate Message Detection
monitor-alert-threshold	Integer	-1	No	No	Threshold in Number of Messages after which the Monitor Swiftlet sends a Queue Monitor Alert

<swiftlet name="sys$queuemanager">
  <queues>
    <queue name="..."/>
  </queues>
</swiftlet>

Related Documents

• Queue Manager Swiftlet (CE)