Store Swiftlet

Overview

The Store Swiftlet is responsible for the persistent and non-persistent message storage in the SwiftMQ router. It manages the underlying database (page.db), transaction log, swap files, and backup/restore operations, ensuring message durability, transactional integrity, and efficient cache management. The Swiftlet supports both persistent queues (disk-based) and non-persistent queues (swap-based), as well as durable subscriber state.

Features

Persistent Message Store (page.db)

The Store Swiftlet manages a disk-based database file (page.db) that stores persistent messages for all queues. Each queue is represented by a separate index within the database, allowing efficient retrieval and deletion of messages. The database is organized into pages, with configurable page sizes and cache settings. The Store Swiftlet ensures ACID properties for persistent messages via a transactional log and recovery mechanisms. On startup, the Swiftlet can perform a compact or resize operation on page.db depending on configuration and recommended page size, optimizing storage efficiency.

Page Size Management

The database page size is configurable and can be tuned for optimal performance. The Swiftlet can recommend a new page size based on message size distribution (via scan), and can automatically resize or compact the database on startup if configured to do so.

Consistency and Recovery

On startup, the Swiftlet performs a recovery process using the transaction log to ensure all committed transactions are applied and incomplete ones are rolled back. It also performs a consistency check on the store, correcting inconsistencies and logging any data loss.

Configuration Example:

<swiftlet name="sys$store">
  <database path="store/page.db" initial-db-size-pages="51200"/>
</swiftlet>

Transaction Log and Checkpointing

All modifications to the persistent store are recorded in a transaction log (transaction.log). The log ensures durability and supports crash recovery. The Swiftlet performs periodic checkpoints, flushing the in-memory cache to disk and truncating the transaction log. The checkpoint interval is determined by the configured log size threshold. The force-sync option can be enabled to guarantee that all log writes are physically synced to disk, reducing the risk of data loss in case of a crash.

XA Transaction Support

The Store Swiftlet supports XA (distributed) transactions. Prepared transactions are stored in a separate log (xa.log or an internal queue), ensuring proper two-phase commit semantics and recovery.

Configuration Example:

<swiftlet name="sys$store">
  <transaction-log path="store/transaction.log" checkpoint-size="209715200" force-sync="true"/>
</swiftlet>

Cache Management

The Store Swiftlet uses an in-memory cache to hold recently accessed pages from the database. The cache size is configurable with minimum and maximum page counts. The cache manager uses an LRU (Least Recently Used) eviction policy for unpinned pages and ensures that dirty pages are flushed to disk during checkpoints. Proper tuning of cache size can significantly improve performance.

Dynamic Cache Sizing

The minimum and maximum cache sizes can be adjusted, with validation to ensure that the minimum does not exceed the maximum and vice versa.

Configuration Example:

<swiftlet name="sys$store">
  <cache min-size="2048" max-size="4096"/>
</swiftlet>

Swap Files for Non-Persistent Queues

Non-persistent queues use swap files for message overflow when memory limits are reached. Swap files are created in the configured swap path, with a configurable roll-over size to limit individual file sizes. When a swap file reaches its roll-over size, a new file is created. Swap files are automatically deleted when the queue is closed. This mechanism ensures that non-persistent messages do not exhaust system memory.

Efficient Message Access

Each swap file maintains its own message index, allowing efficient retrieval and deletion of messages by file pointer.

Configuration Example:

<swiftlet name="sys$store">
  <swap path="store/swap" roll-over-size="20971520"/>
</swiftlet>

Backup and Restore

The Store Swiftlet provides built-in backup capabilities, allowing administrators to create save sets containing the current state of the persistent store, transaction log, and durable subscriber data. Backups are stored in the configured backup path, and a configurable number of backup generations are retained. The backup process is coordinated with checkpointing to ensure consistency. The Swiftlet also manages backup metadata and file lists for each save set.

Backup Generation Management

The number of backup generations to keep is configurable. When the limit is exceeded, the oldest backups are automatically deleted.

Backup Path Validation

The backup path is validated for existence and writability. Changes to the backup path or generation count trigger a rescan of available save sets.

Configuration Example:

<swiftlet name="sys$store">
  <backup path="backups" keep-generations="5"/>
</swiftlet>

Durable Subscriber Store

The Swiftlet maintains a separate store for durable subscriber information, including topic name, selector, and no-local flag. Durable subscriber state is stored in files under the configured path, using the naming pattern <clientId>$<durableName>.durable. This ensures that durable subscriptions survive router restarts and can be recovered after failure.

Configuration Example:

<swiftlet name="sys$store">
  <durable-subscriber path="store/durables"/>
</swiftlet>

Maintenance Operations: Shrink and Scan

The Store Swiftlet supports maintenance operations to optimize and analyze the persistent store. The shrink operation reclaims unused space in the database, reducing file size. The scan operation analyzes message sizes in the store and recommends an optimal page size for future operation. Both operations are coordinated with checkpointing to ensure data consistency and can be triggered via management commands or scheduled jobs.

Automated and Manual Execution

Shrink and scan operations can be triggered manually via management commands or scheduled as jobs using the scheduler Swiftlet integration.

Internal Queue Naming

<clientId>$<durableName> — Queue name pattern for durable subscription state storage.

Configuration Guide

Increase Cache Size for High Throughput

To improve message throughput and reduce disk I/O, increase the cache size for the persistent store. This is useful when the router handles a high volume of persistent messages and sufficient memory is available.

Edit the routerconfig.xml to increase the values of the min-size and max-size attributes under the cache entity.
Restart the router for the changes to take effect.

<swiftlet name="sys$store">
  <cache min-size="4096" max-size="8192"/>
</swiftlet>

Configure Store Backup with Custom Path and Generations

Set up automated backups of the persistent store to a specific directory, retaining a defined number of backup generations. This is recommended for production environments to ensure recoverability.

Set the path and keep-generations attributes under the backup entity in routerconfig.xml.
Ensure the backup path is writable by the router process.
Backups can be triggered via management commands or scheduled jobs.

<swiftlet name="sys$store">
  <backup path="/var/swiftmq/backups" keep-generations="7"/>
</swiftlet>

Enable Force Sync for Maximum Durability

Enable the force-sync option on the transaction log to guarantee that all log writes are physically synced to disk, minimizing the risk of data loss in case of a crash. This is recommended for environments with strict durability requirements.

Set the force-sync attribute to true under the transaction-log entity in routerconfig.xml.
Restart the router for the change to take effect.

<swiftlet name="sys$store">
  <transaction-log force-sync="true"/>
</swiftlet>

Resize or Compact the Database on Startup

Automatically resize the database page size or compact the store on startup, based on the recommended page size. This helps optimize storage and performance after a scan operation.

Set perform-resize-on-startup or perform-compact-on-startup to true under the database entity in routerconfig.xml.
Restart the router. The store will resize or compact as needed.

<swiftlet name="sys$store">
  <database perform-resize-on-startup="true"/>
</swiftlet>

CLI Commands

backup

Description: Perform Backup Now

Usage: backup

shrink

Description: Perform Shrink Now

Usage: shrink

scan

Description: Scan page.db

Usage: scan

Scheduler Jobs

Backup

Description: Performs a Backup of the persistent Store

Scan

Description: Scans page.db and recommends a new page size

Shrink

Description: Performs a Shrink of the page.db