Skip to main content

47. Dedicated BullMQ Queue for Admin Tasks

Status: Accepted Date: 2025-07-06

Context​

The Mercury system requires a mechanism to execute privileged, administrative tasks (e.g., pausing a tournament, issuing a refund, recalculating user statistics). These operations are sensitive, potentially long-running, and must not interfere with the primary, time-sensitive trading operations. Running these tasks synchronously in a request-response cycle or mixing them in the same job queues as trading logic could lead to performance bottlenecks, resource contention, and a lack of proper isolation.

Decision​

We will implement a dedicated, separate BullMQ message queue specifically for administrative tasks. This queue will be named admin-jobs.

Key characteristics of this approach:

  1. Isolation: The admin-jobs queue will have its own dedicated set of worker processes. This ensures that the execution of admin tasks does not consume resources (CPU, memory, network) allocated to the core trading logic queues.
  2. Controlled Access: The ability to add jobs to the admin-jobs queue will be tightly controlled and exposed only through a secure administrative service or CLI. It will not be accessible from the main application services.
  3. Auditing: All jobs added to this queue represent a specific administrative action, creating a clear and auditable trail of privileged operations.

Consequences​

Positive:

  • Improved Reliability & Stability: Isolates critical trading queues from potentially resource-intensive or faulty admin tasks, improving the overall stability of the system.
  • Enhanced Security & Auditing: Centralizes all privileged operations into a single, controllable channel, making it easier to secure, monitor, and audit.
  • Scalability: The processing capacity for administrative tasks can be scaled independently by adding more admin-jobs workers, without affecting the core trading system.
  • Asynchronous Operations: Allows administrators to trigger long-running tasks without blocking the UI or an API call, improving the user experience of admin interfaces.

Negative:

  • Increased Infrastructure Overhead: Requires running and managing a separate set of queue workers, which adds to the operational complexity and resource footprint.
  • Slightly More Complex Logic: Code that initiates an admin task needs to know how to correctly format and enqueue a job into the admin-jobs queue rather than just calling a service method directly.

Mitigation:

  • Containerization: The additional workers can be managed easily as another container within our Docker/Kubernetes environment, minimizing the operational overhead.
  • Admin Service Abstraction: We will create a dedicated AdminService that abstracts away the details of enqueuing jobs. Other parts of the system will simply call methods on this service (e.g., adminService.pauseTournament(tournamentId)), and the service itself will handle the job creation and queueing. This keeps the rest of the codebase clean.