Store Swiftlet (HA)
Overview
The High Availability (HA) Store Swiftlet extends the standard CE Store Swiftlet to provide synchronous replication of all persistent data for active/standby failover. It ensures that all persistent operations are synchronously mirrored to a standby node, enabling seamless failover without data loss. For standard store features such as swap files, backup, shrink, and cache management, refer to the CE Store Swiftlet documentation.
Features
Synchronous Replication of Persistent Store
In HA mode, all persistent data—including the database, transaction log, and durable subscription store—are synchronously replicated from the active node to the standby node. This is achieved by intercepting all write, free, and truncate operations and forwarding them to the standby via a dedicated replication protocol. The replication is transactionally consistent: a store operation is not acknowledged to the client until it has been successfully replicated to the standby. This guarantees zero data loss in the event of a failover.
The replication protocol covers all persistent store files and operations, including the initial transfer of the full store image when a standby joins, and ongoing mirroring of all subsequent changes. The protocol uses a series of requests (e.g., FileStartRequest, FileChunkRequest, PageDBPutRequest, TxLogWriteRequest) to transfer both file data and logical operations. The standby node applies these operations in real time to maintain an up-to-date copy of the store.
Store Image Transfer on Standby Join
When a standby node joins, the active node sends the complete image of the persistent store (database, transaction log, and durable subscriber files) using chunked file transfer requests. This ensures the standby is fully synchronized before entering replication mode.
Transactional Consistency
All store operations are coordinated so that changes are only committed on the active node after the corresponding replication request has been acknowledged by the standby. This prevents split-brain scenarios and ensures data consistency across failover.
Persistent Store Options for HA
The HA Store Swiftlet supports three persistent store deployment options, each with different trade-offs for performance, reliability, and operational complexity:
- Replicated File Store: Each node has its own local disk. All persistent data is synchronously replicated from the active to the standby node. This is the default and most common mode.
- Shared File Store: Both nodes access the same shared storage (e.g., SAN, NAS). Replication is not required, but only one node can be active at a time. The
shared-storeproperty must be set totrue. - Shared JDBC Store: Both nodes use the same JDBC-accessible database for persistence. As with shared file store, only one node is active at a time, and
shared-storemust betrue.
In shared store modes, the replication protocol is bypassed and direct disk or database access is used. In replicated mode, all data is mirrored over the HA channel.
Replicated File Store
Default mode. Each node uses its own local disk. All changes are synchronously replicated from active to standby.
Shared File Store
Both nodes access a shared filesystem. Set shared-store="true" in the configuration. Replication is not used; failover is handled by switching active access to the shared files.
Shared JDBC Store
Both nodes use the same JDBC database for persistence. Set shared-store="true". Replication is not used; failover is handled by switching active access to the shared database.
Configuration Example:
<swiftlet name="sys$store" shared-store="true"/>
Force Sync Behavior in Standalone Mode
The force-sync-in-standalone-mode property controls whether the transaction log is synchronously flushed to disk when the HA instance is running in STANDALONE mode (i.e., not actively replicating to a standby). By default, this is enabled (true) to maximize data safety. If disabled, performance may improve at the cost of potential data loss in the event of a crash while running standalone.
Dynamic Adjustment
The force-sync-in-standalone-mode property can be changed at runtime and is monitored by the store logic. When enabled, the transaction log is always force-synced to disk in standalone mode; when disabled, it may be buffered.
Configuration Example:
<swiftlet name="sys$store">
<transaction-log force-sync-in-standalone-mode="false"/>
</swiftlet>
HA State Management and Store Role Switching
The HA Store Swiftlet automatically manages its role (active, standby, or standalone) in coordination with the HA Controller Swiftlet. When the node becomes active, it initializes replication and ensures all persistent operations are mirrored to the standby. When entering standby, it prepares to receive replicated data and applies all incoming changes. If the node transitions to standalone (e.g., due to loss of HA connectivity), it disables replication and can optionally force sync all writes to disk for safety, depending on the force-sync-in-standalone-mode setting.
Automatic Store Flushing and Image Transfer
On role transitions, the store is flushed and, if necessary, the full store image is transferred to the standby to ensure consistency.
Configuration Guide
Enable Shared Store Mode for HA
Use this scenario when both HA nodes have access to a shared filesystem or JDBC database. This disables replication and allows both nodes to access the same persistent store, but only one node should be active at a time.
- Set the
shared-storeattribute totrueon thesys$storeswiftlet. - Ensure both nodes are configured to use the same database or filesystem paths.
<swiftlet name="sys$store" shared-store="true"/>
Disable Force Sync in Standalone Mode
If you want to optimize performance when the node is running in standalone mode (not replicating), you can disable force sync. This reduces disk I/O but increases risk of data loss if the node crashes while not in HA mode.
- Add or modify the
transaction-logentity under thesys$storeswiftlet. - Set the
force-sync-in-standalone-modeattribute tofalse.
<swiftlet name="sys$store">
<transaction-log force-sync-in-standalone-mode="false"/>
</swiftlet>
Configuration Reference
The top-level entity in routerconfig.xml is <swiftlet name="sys$store">.
<swiftlet name="sys$store"> Properties
These properties are attributes of the <swiftlet name="sys$store"> entity.
| Parameter | Type | Default | Mandatory | Reboot Required | Description |
|---|---|---|---|---|---|
shared-store |
Boolean | false |
No | Yes | Use a shared Store |
<swiftlet name="sys$store" shared-store="false"/>
<backup> Entity
Backup Settings
This is a fixed child entity of <swiftlet name="sys$store">.
| Parameter | Type | Default | Mandatory | Reboot Required | Description |
|---|---|---|---|---|---|
path |
String | ./ |
No | No | Root Path for Backup Save Sets |
keep-generations |
Integer | 3 |
No | No | Number of Generations to keep (min: 1) |
<swiftlet name="sys$store">
<backup path="..." keep-generations="..."/>
</swiftlet>
<database> Entity
Database Settings
This is a fixed child entity of <swiftlet name="sys$store">.
| Parameter | Type | Default | Mandatory | Reboot Required | Description |
|---|---|---|---|---|---|
path |
String | ./ |
No | Yes | Path of the Database (page.db) |
initial-db-size-pages |
Integer | 25600 |
No | Yes | Initial Size of the Database in Pages (min: 10) |
size-collect-interval |
Long | 1000 |
No | No | Interval to collect the Usage |
page-size-current |
Integer | 2048 |
No | No | Current Size of the Database Pages |
page-size-recommended |
Integer | 2048 |
No | No | Recommended Size of the Database Pages |
page-size-max |
Integer | 32768 |
No | No | Maximum Page Size (min: 2048) |
perform-compact-on-startup |
Boolean | false |
No | No | Perform a compact of the page.db on startup if current page size is equal to recommended page size |
perform-resize-on-startup |
Boolean | false |
No | No | Perform a page resize on startup if the recommended size is greater than the current page size |
<swiftlet name="sys$store">
<database path="..." initial-db-size-pages="..." size-collect-interval="..." page-size-current="..." page-size-recommended="..." page-size-max="..." perform-compact-on-startup="..." perform-resize-on-startup="..."/>
</swiftlet>
<cache> Entity
Cache Settings
This is a fixed child entity of <swiftlet name="sys$store">.
| Parameter | Type | Default | Mandatory | Reboot Required | Description |
|---|---|---|---|---|---|
min-size |
Integer | 1024 |
No | Yes | Min. Size (Pages) (min: 512) |
max-size |
Integer | 2048 |
No | Yes | Max. Size (Pages) (min: 512) |
<swiftlet name="sys$store">
<cache min-size="..." max-size="..."/>
</swiftlet>
<transaction-log> Entity
Transaction Log Settings
This is a fixed child entity of <swiftlet name="sys$store">.
| Parameter | Type | Default | Mandatory | Reboot Required | Description |
|---|---|---|---|---|---|
path |
String | ./ |
No | Yes | Path of the Transaction Log (transaction.log) |
checkpoint-size |
Long | 104857600 |
No | Yes | Size at which a Checkpoint is performed (min: 1048576) |
force-sync |
Boolean | false |
No | Yes | Force a sync with the disk |
force-sync-in-standalone-mode |
Boolean | true |
No | No | Force a sync with the disk while the HA instance is in STANDALONE mode |
<swiftlet name="sys$store">
<transaction-log path="..." checkpoint-size="..." force-sync="..." force-sync-in-standalone-mode="..."/>
</swiftlet>
<swap> Entity
Swap Settings
This is a fixed child entity of <swiftlet name="sys$store">.
| Parameter | Type | Default | Mandatory | Reboot Required | Description |
|---|---|---|---|---|---|
path |
String | ./ |
No | Yes | Path of Swap Files |
roll-over-size |
Long | 10485760 |
No | Yes | Roll Over Size (min: 1048576) |
<swiftlet name="sys$store">
<swap path="..." roll-over-size="..."/>
</swiftlet>
<durable-subscriber> Entity
Durable Subscriber Settings
This is a fixed child entity of <swiftlet name="sys$store">.
| Parameter | Type | Default | Mandatory | Reboot Required | Description |
|---|---|---|---|---|---|
path |
String | ./ |
No | Yes | Path of Durable Subscriber Files |
<swiftlet name="sys$store">
<durable-subscriber path="..."/>
</swiftlet>