Building a Production Job Queue with BullMQ — Implementing Priority, Retry Backoff, and Dead Letter Queue in TypeScript

When an API handler directly handles email sending or payment requests, delays from external services translate directly into HTTP timeouts. An async job queue breaks this coupling. The API responds immediately after enqueuing the job, and actual processing is handled by background workers.

But scenarios where simply "enqueuing" isn't enough emerge quickly. Payments and marketing newsletters shouldn't compete at the same priority, retrying every second when an external API returns 503 adds more pressure to an already-stressed server, and silently discarding jobs that have exhausted all retries leaves no way to analyze the cause later.

This article covers how to implement these three patterns — priority queues, exponential backoff retries, and Dead Letter Queues — with BullMQ and TypeScript.

Things to Consider When Choosing BullMQ

BullMQ is a Node.js job queue library that uses Redis as its store and is a complete rewrite of the original Bull. It was designed with TypeScript in mind from the start and minimizes job loss with atomic operations based on Lua scripts.

Item	Rating	Notes
TypeScript support	★★★★★	Full job data type safety via generics
Redis-based performance	★★★★★	High throughput even on a single instance
Feature richness	★★★★★	Priority, delay, repeat, rate limiting, DAG flows
Reliability	★★★★☆	Lua script atomic operations, at-least-once guarantee
Operational complexity	★★★☆☆	Requires separate Redis provisioning and monitoring
Ecosystem	★★★★★	Official NestJS support, OpenTelemetry, Bull Board
Built-in DLQ	★★☆☆☆	Requires custom implementation — the flexibility is actually a strength

PostgreSQL-based alternatives (pgboss, Graphile Worker, etc.) have the advantage of leveraging your existing DB infrastructure. Worth comparing if adding Redis infrastructure feels burdensome. If you're already using Redis or throughput matters, BullMQ shines.

Core Concepts

BullMQ Architecture

BullMQ consists of three core components:

Queue: Interface for adding jobs and managing configuration
Worker: Processor that actually consumes and handles jobs
QueueEvents: Subscribes to job state change events (completed, failed, progress, etc.)

FlowProducer (parent-child dependency DAG pipelines) also exists but is not covered in this article.

Diagram 1

DLQ is not a built-in BullMQ feature. As shown by the dashed lines, you implement the pattern yourself — subscribing to the failed event and transferring jobs to a separate queue.

Installation:

bash

pnpm add bullmq

Separating the Redis connection config allows it to be reused across both Queue and Worker.

typescript

// config/redis.ts
import { ConnectionOptions } from 'bullmq'
 
export const connection: ConnectionOptions = {
  host: process.env.REDIS_HOST ?? 'localhost',
  port: Number(process.env.REDIS_PORT ?? 6379),
}

Priority Queue

BullMQ priorities are specified as numbers in the range 1–2,097,152, where lower numbers mean higher priority.

typescript

// queues/notification.queue.ts
import { Queue } from 'bullmq'
import { connection } from '../config/redis'
 
export interface NotificationJobData {
  userId: string
  type: 'urgent' | 'normal' | 'batch'
  message: string
}
 
export const notificationQueue = new Queue<NotificationJobData>('notifications', {
  connection,
  defaultJobOptions: {
    removeOnComplete: { count: 1000, age: 3600 },
    removeOnFail: { count: 5000 },
  },
})
 
// Priority 1: security alerts that require immediate processing
await notificationQueue.add(
  'urgent-alert',
  { userId: 'u-123', type: 'urgent', message: 'Account security alert' },
  { priority: 1 }
)
 
// Priority 10: regular user notifications
await notificationQueue.add(
  'normal-notification',
  { userId: 'u-456', type: 'normal', message: 'A new comment has been posted' },
  { priority: 10 }
)
 
// Priority 100: batch marketing emails (lowest priority)
await notificationQueue.add(
  'batch-email',
  { userId: 'u-789', type: 'batch', message: 'This week\'s newsletter' },
  { priority: 100 }
)

Even after a job has been enqueued, you can change its priority dynamically with changePriority(). This API is useful if you have a UI where users can mark certain tasks as "urgent."

typescript

const job = await notificationQueue.getJob(jobId)
if (job) {
  await job.changePriority({ priority: 1 })
}

Retry Backoff

Retrying immediately when an external API temporarily returns 503 stacks more requests onto an already-stressed server. Exponential backoff doubles the wait time with each retry, giving the external service time to recover.

typescript

await paymentQueue.add(
  'process-payment',
  {
    orderId: 'order-001',
    amount: 15000,
    currency: 'KRW',
    idempotencyKey: crypto.randomUUID(),
  },
  {
    attempts: 5,
    backoff: {
      type: 'exponential',
      delay: 1000,  // 1s → 2s → 4s → 8s → 16s
    },
  }
)

If you need different wait times depending on the error content — like Rate Limit headers — use a custom backoff function.

typescript

const worker = new Worker('api-calls', processor, {
  connection,
  settings: {
    backoffStrategy: (attemptsMade, type, err, job) => {
      // In TypeScript strict mode, err is unknown — type guard required
      if (err instanceof Error && err.message.includes('Rate limit')) {
        return 60_000 * attemptsMade  // Increases in minutes for rate limits
      }
      return 2 ** attemptsMade * 1000
    },
  },
})

Dead Letter Queue

How to handle jobs that have exhausted all retries is the core of DLQ. Unlike simple failure logging, a DLQ is a space for root cause analysis and manual reprocessing.

Diagram 2

The implementation pattern is straightforward. Check the job.attemptsMade >= maxAttempts condition in the Worker's failed event, and only then add the job metadata to a separate DLQ queue.

typescript

// Core pattern — see the "Production Implementation" section below for the full implementation
const dlqQueue = new Queue('payments-dlq', { connection })
 
paymentWorker.on('failed', async (job, err) => {
  if (!job) return  // BullMQ can pass job as undefined in the failed event
 
  const maxAttempts = job.opts.attempts ?? 1
  if (job.attemptsMade >= maxAttempts) {
    await dlqQueue.add('failed-payment', {
      originalJobId: job.id ?? 'unknown',
      originalData: job.data,
      errorMessage: err instanceof Error ? err.message : String(err),
      failedAt: new Date().toISOString(),
    })
  }
})

Production Implementation — Payment Processing Queue

A complete example tying together all three concepts. From file structure to DLQ reprocessing utilities, this shows the structure used in production.

Queue Definition

typescript

// queues/payment.queue.ts
import { Queue } from 'bullmq'
import { connection } from '../config/redis'
 
export interface PaymentJobData {
  orderId: string
  amount: number
  currency: string
  idempotencyKey: string
}
 
export const paymentQueue = new Queue<PaymentJobData>('payments', {
  connection,
  defaultJobOptions: {
    attempts: 5,
    backoff: { type: 'exponential', delay: 1000 },
    removeOnComplete: { count: 500, age: 3600 },
    removeOnFail: { count: 2000 },
  },
})

DLQ Queue Definition (Separated into Its Own File)

Separating the DLQ into its own file lets you configure DLQ-specific retention policies, monitoring, and workers independently.

typescript

// queues/payment-dlq.queue.ts
import { Queue } from 'bullmq'
import { connection } from '../config/redis'
import type { PaymentJobData } from './payment.queue'
 
export interface DLQJobData {
  originalJobId: string
  originalJobName: string
  originalData: PaymentJobData
  errorMessage: string
  attemptsMade: number
  failedAt: string
}
 
export const paymentDLQQueue = new Queue<DLQJobData>('payments-dlq', { connection })

Worker and DLQ Transfer

typescript

// workers/payment.worker.ts
import { Worker } from 'bullmq'
import { connection } from '../config/redis'
import type { PaymentJobData } from '../queues/payment.queue'
import { paymentDLQQueue } from '../queues/payment-dlq.queue'
 
const paymentWorker = new Worker<PaymentJobData>(
  'payments',
  async (job) => {
    await job.updateProgress(10)
 
    // callPaymentGateway is the actual payment gateway client — implementation omitted
    const result = await callPaymentGateway({
      orderId: job.data.orderId,
      amount: job.data.amount,
      idempotencyKey: job.data.idempotencyKey,
    })
 
    await job.updateProgress(100)
    return result
  },
  {
    connection,
    concurrency: 5,
    limiter: {
      max: 10,       // Max 10 requests per second — handles payment gateway rate limits
      duration: 1000,
    },
  }
)
 
paymentWorker.on('failed', async (job, err) => {
  if (!job) return
 
  const maxAttempts = job.opts.attempts ?? 1
 
  if (job.attemptsMade >= maxAttempts) {
    console.error(`[DLQ] Job ${job.id} finally failed → transferring to DLQ`)
 
    await paymentDLQQueue.add(
      'failed-payment',
      {
        originalJobId: job.id ?? 'unknown',
        originalJobName: job.name,
        originalData: job.data,
        errorMessage: err instanceof Error ? err.message : String(err),
        attemptsMade: job.attemptsMade,
        failedAt: new Date().toISOString(),
      },
      {
        removeOnComplete: { age: 7 * 24 * 3600 },
        // If a worker is added to the DLQ for auto-reprocessing, prevents records from being auto-deleted even if that processing fails
        removeOnFail: false,
      }
    )
  }
})
 
paymentWorker.on('completed', (job) => {
  console.log(`[${job.id}] Payment completed: ${job.data.orderId}`)
})

DLQ Reprocessing Utility

A script for manually reprocessing DLQ jobs after the root cause has been resolved.

typescript

// scripts/reprocess-dlq.ts
import { Queue } from 'bullmq'
import { connection } from '../config/redis'
import type { DLQJobData } from '../queues/payment-dlq.queue'
 
const dlqQueue = new Queue<DLQJobData>('payments-dlq', { connection })
const paymentQueue = new Queue('payments', { connection })
 
async function listDLQJobs() {
  // Without a DLQ worker, jobs accumulate in 'waiting' state
  const jobs = await dlqQueue.getJobs(['waiting', 'delayed'])
  return jobs.map((job) => ({
    id: job.id,
    orderId: job.data.originalData.orderId,
    error: job.data.errorMessage,
    attempts: job.data.attemptsMade,
    failedAt: job.data.failedAt,
  }))
}
 
async function reprocessDLQJob(dlqJobId: string) {
  const dlqJob = await dlqQueue.getJob(dlqJobId)
  if (!dlqJob) throw new Error(`DLQ job ${dlqJobId} not found`)
 
  const newJob = await paymentQueue.add(
    dlqJob.data.originalJobName,
    dlqJob.data.originalData,
    {
      attempts: 3,
      backoff: { type: 'exponential', delay: 2000 },
      priority: 5,  // Reprocessed jobs get higher priority than regular jobs
    }
  )
 
  await dlqJob.remove()
 
  console.log(`Reprocessing complete: DLQ ${dlqJobId} → new job ${newJob.id}`)
  return newJob.id
}

Common Mistakes

Misconfiguring Redis `maxmemory-policy`

This is a silent and critical mistake. When Redis is shared with a cache and an eviction policy like allkeys-lru is active, job data being processed can be arbitrarily deleted when memory runs low.

Separate the Redis instance for BullMQ and always apply this setting.

bash

# redis.conf
maxmemory-policy noeviction

Not Setting `removeOnComplete` / `removeOnFail`

Completed jobs are not automatically removed. Specifying both count and age keeps memory usage predictable regardless of traffic patterns.

typescript

// count only: jobs from months ago can linger during low-traffic periods
// age only: memory can spike sharply during traffic surges
// Both together:
removeOnComplete: { count: 1000, age: 3600 }
removeOnFail: { count: 5000 }

Not Ensuring Idempotency in Processor Functions

BullMQ follows at-least-once semantics. If a worker crashes while processing a job, that job will be reprocessed. For operations with side effects like payments or email sending, DB-level deduplication is essential.

typescript

// Example using Prisma
async function paymentProcessor(job: Job<PaymentJobData>) {
  const existing = await prisma.payment.findUnique({
    where: { idempotencyKey: job.data.idempotencyKey },
  })
 
  if (existing) {
    return existing  // Already processed payment: return previous result
  }
 
  return await prisma.payment.create({ data: { ...job.data } })
}

Overlooking the Concurrency Setting

The default concurrency for a Worker is 1. If throughput is lower than expected, check this setting first.

Diagram 3

For CPU-intensive work, tune based on core count; for I/O-intensive work, tune with load testing in the 20–50 range.

NestJS Integration

Using the @nestjs/bullmq package, you can modularize the same patterns with a decorator-based approach.

typescript

// notifications/notifications.module.ts
import { BullModule } from '@nestjs/bullmq'
import { Module } from '@nestjs/common'
import { NotificationProcessor } from './notification.processor'
import { NotificationService } from './notification.service'
 
@Module({
  imports: [
    BullModule.registerQueue({
      name: 'notifications',
      defaultJobOptions: {
        attempts: 3,
        backoff: { type: 'exponential', delay: 1000 },
        removeOnComplete: { count: 1000, age: 3600 },
        removeOnFail: { count: 5000 },
      },
    }),
  ],
  providers: [NotificationService, NotificationProcessor],
  exports: [NotificationService],
})
export class NotificationsModule {}

typescript

// notifications/notification.processor.ts
import { Processor, WorkerHost, OnWorkerEvent } from '@nestjs/bullmq'
import { Job } from 'bullmq'
import type { NotificationJobData } from './notification.types'
 
@Processor('notifications', { concurrency: 10 })
export class NotificationProcessor extends WorkerHost {
  async process(job: Job<NotificationJobData>): Promise<void> {
    switch (job.name) {
      case 'send-email':
        // sendEmail, sendPush are actual notification service clients — implementation omitted
        await this.sendEmail(job.data)
        break
      case 'send-push':
        await this.sendPush(job.data)
        break
      default:
        throw new Error(`Unknown job type: ${job.name}`)
    }
  }
 
  @OnWorkerEvent('failed')
  onFailed(job: Job<NotificationJobData> | undefined, err: Error) {
    if (!job) return  // job can be undefined in the failed event
    console.error(`Notification job failed [${job.id}]: ${err.message}`)
  }
}

Monitoring: Bull Board

@bull-board is the most widely used monitoring UI in the BullMQ ecosystem. You can view job status per queue, failure reasons, and retry history in real time, and retry or delete jobs directly from the UI.

typescript

// app.ts (Express-based)
import { createBullBoard } from '@bull-board/api'
import { BullMQAdapter } from '@bull-board/api/bullMQAdapter'
import { ExpressAdapter } from '@bull-board/express'
import { notificationQueue } from './queues/notification.queue'
import { paymentQueue } from './queues/payment.queue'
import { paymentDLQQueue } from './queues/payment-dlq.queue'
 
const serverAdapter = new ExpressAdapter()
serverAdapter.setBasePath('/admin/queues')
 
createBullBoard({
  queues: [
    new BullMQAdapter(notificationQueue),
    new BullMQAdapter(paymentQueue),
    new BullMQAdapter(paymentDLQQueue),
  ],
  serverAdapter,
})
 
app.use('/admin/queues', serverAdapter.getRouter())

Summary

Key patterns in summary:

Priority queue: Lower numbers are processed first. Can be changed at runtime with changePriority()
Retry backoff: The exponential strategy is most effective for external service outages. Custom functions can handle special cases like Rate Limits (but err instanceof Error type guard is required)
DLQ: Implemented manually using the failed event + attemptsMade >= attempts condition. Must be separated into its own queue to configure retention policies and monitoring independently

Three things to get right first in production:

Redis noeviction setting: Sharing an instance with an active eviction policy causes job data to silently disappear
Specify both count and age for removeOnComplete: Using either alone makes memory usage unpredictable depending on traffic patterns
Idempotency keys: With at-least-once guarantees, operations with side effects require DB-level deduplication beyond the queue layer

A good starting point is to set up a single, simple email-sending queue with attempts: 3 plus exponential backoff, visually verify the job flow with Bull Board, then progressively add priority and DLQ.

References

BullMQ Official Documentation — Complete guide to priority, retries, concurrency, and rate limiting
Prioritized Jobs | BullMQ Docs — Official priority queue documentation
Retrying Failing Jobs | BullMQ Docs — Official retry strategy documentation
BullMQ GitHub Release Notes — v5 series changelog
NestJS + BullMQ: Top 4 Use Cases & Quick Tutorial — NestJS integration practical examples
BullMQ Architecture for High Traffic — High-traffic architecture design

#BullMQ#TypeScript#Redis#Dead Letter Queue#우선순위 큐#백오프 재시도

Building a Production Job Queue with BullMQ — Implementing Priority, Retry Backoff, and Dead Letter Queue in TypeScript | DEV BAK - 기술블로그

Backend

Building a Production Job Queue with BullMQ — Implementing Priority, Retry Backoff, and Dead Letter Queue in TypeScript

This article covers how to implement these three patterns — priority queues, exponential backoff retries, and Dead Letter Queues — with BullMQ and TypeScript.

Things to Consider When Choosing BullMQ

Item	Rating	Notes
TypeScript support	★★★★★	Full job data type safety via generics
Redis-based performance	★★★★★	High throughput even on a single instance
Feature richness	★★★★★	Priority, delay, repeat, rate limiting, DAG flows
Reliability	★★★★☆	Lua script atomic operations, at-least-once guarantee
Operational complexity	★★★☆☆	Requires separate Redis provisioning and monitoring
Ecosystem	★★★★★	Official NestJS support, OpenTelemetry, Bull Board
Built-in DLQ	★★☆☆☆	Requires custom implementation — the flexibility is actually a strength

Core Concepts

BullMQ Architecture

BullMQ consists of three core components:

Queue: Interface for adding jobs and managing configuration
Worker: Processor that actually consumes and handles jobs
QueueEvents: Subscribes to job state change events (completed, failed, progress, etc.)

FlowProducer (parent-child dependency DAG pipelines) also exists but is not covered in this article.

Diagram 1

DLQ is not a built-in BullMQ feature. As shown by the dashed lines, you implement the pattern yourself — subscribing to the failed event and transferring jobs to a separate queue.

Installation:

bash

pnpm add bullmq

Separating the Redis connection config allows it to be reused across both Queue and Worker.

typescript

// config/redis.ts
import { ConnectionOptions } from 'bullmq'
 
export const connection: ConnectionOptions = {
  host: process.env.REDIS_HOST ?? 'localhost',
  port: Number(process.env.REDIS_PORT ?? 6379),
}

Priority Queue

BullMQ priorities are specified as numbers in the range 1–2,097,152, where lower numbers mean higher priority.

typescript

// queues/notification.queue.ts
import { Queue } from 'bullmq'
import { connection } from '../config/redis'
 
export interface NotificationJobData {
  userId: string
  type: 'urgent' | 'normal' | 'batch'
  message: string
}
 
export const notificationQueue = new Queue<NotificationJobData>('notifications', {
  connection,
  defaultJobOptions: {
    removeOnComplete: { count: 1000, age: 3600 },
    removeOnFail: { count: 5000 },
  },
})
 
// Priority 1: security alerts that require immediate processing
await notificationQueue.add(
  'urgent-alert',
  { userId: 'u-123', type: 'urgent', message: 'Account security alert' },
  { priority: 1 }
)
 
// Priority 10: regular user notifications
await notificationQueue.add(
  'normal-notification',
  { userId: 'u-456', type: 'normal', message: 'A new comment has been posted' },
  { priority: 10 }
)
 
// Priority 100: batch marketing emails (lowest priority)
await notificationQueue.add(
  'batch-email',
  { userId: 'u-789', type: 'batch', message: 'This week\'s newsletter' },
  { priority: 100 }
)

Even after a job has been enqueued, you can change its priority dynamically with changePriority(). This API is useful if you have a UI where users can mark certain tasks as "urgent."

typescript

const job = await notificationQueue.getJob(jobId)
if (job) {
  await job.changePriority({ priority: 1 })
}

Retry Backoff

typescript

await paymentQueue.add(
  'process-payment',
  {
    orderId: 'order-001',
    amount: 15000,
    currency: 'KRW',
    idempotencyKey: crypto.randomUUID(),
  },
  {
    attempts: 5,
    backoff: {
      type: 'exponential',
      delay: 1000,  // 1s → 2s → 4s → 8s → 16s
    },
  }
)

If you need different wait times depending on the error content — like Rate Limit headers — use a custom backoff function.

typescript

const worker = new Worker('api-calls', processor, {
  connection,
  settings: {
    backoffStrategy: (attemptsMade, type, err, job) => {
      // In TypeScript strict mode, err is unknown — type guard required
      if (err instanceof Error && err.message.includes('Rate limit')) {
        return 60_000 * attemptsMade  // Increases in minutes for rate limits
      }
      return 2 ** attemptsMade * 1000
    },
  },
})

Dead Letter Queue

How to handle jobs that have exhausted all retries is the core of DLQ. Unlike simple failure logging, a DLQ is a space for root cause analysis and manual reprocessing.

Diagram 2

The implementation pattern is straightforward. Check the job.attemptsMade >= maxAttempts condition in the Worker's failed event, and only then add the job metadata to a separate DLQ queue.

typescript

// Core pattern — see the "Production Implementation" section below for the full implementation
const dlqQueue = new Queue('payments-dlq', { connection })
 
paymentWorker.on('failed', async (job, err) => {
  if (!job) return  // BullMQ can pass job as undefined in the failed event
 
  const maxAttempts = job.opts.attempts ?? 1
  if (job.attemptsMade >= maxAttempts) {
    await dlqQueue.add('failed-payment', {
      originalJobId: job.id ?? 'unknown',
      originalData: job.data,
      errorMessage: err instanceof Error ? err.message : String(err),
      failedAt: new Date().toISOString(),
    })
  }
})

Production Implementation — Payment Processing Queue

A complete example tying together all three concepts. From file structure to DLQ reprocessing utilities, this shows the structure used in production.

Queue Definition

typescript

// queues/payment.queue.ts
import { Queue } from 'bullmq'
import { connection } from '../config/redis'
 
export interface PaymentJobData {
  orderId: string
  amount: number
  currency: string
  idempotencyKey: string
}
 
export const paymentQueue = new Queue<PaymentJobData>('payments', {
  connection,
  defaultJobOptions: {
    attempts: 5,
    backoff: { type: 'exponential', delay: 1000 },
    removeOnComplete: { count: 500, age: 3600 },
    removeOnFail: { count: 2000 },
  },
})

DLQ Queue Definition (Separated into Its Own File)

Separating the DLQ into its own file lets you configure DLQ-specific retention policies, monitoring, and workers independently.

typescript

// queues/payment-dlq.queue.ts
import { Queue } from 'bullmq'
import { connection } from '../config/redis'
import type { PaymentJobData } from './payment.queue'
 
export interface DLQJobData {
  originalJobId: string
  originalJobName: string
  originalData: PaymentJobData
  errorMessage: string
  attemptsMade: number
  failedAt: string
}
 
export const paymentDLQQueue = new Queue<DLQJobData>('payments-dlq', { connection })

Worker and DLQ Transfer

typescript

// workers/payment.worker.ts
import { Worker } from 'bullmq'
import { connection } from '../config/redis'
import type { PaymentJobData } from '../queues/payment.queue'
import { paymentDLQQueue } from '../queues/payment-dlq.queue'
 
const paymentWorker = new Worker<PaymentJobData>(
  'payments',
  async (job) => {
    await job.updateProgress(10)
 
    // callPaymentGateway is the actual payment gateway client — implementation omitted
    const result = await callPaymentGateway({
      orderId: job.data.orderId,
      amount: job.data.amount,
      idempotencyKey: job.data.idempotencyKey,
    })
 
    await job.updateProgress(100)
    return result
  },
  {
    connection,
    concurrency: 5,
    limiter: {
      max: 10,       // Max 10 requests per second — handles payment gateway rate limits
      duration: 1000,
    },
  }
)
 
paymentWorker.on('failed', async (job, err) => {
  if (!job) return
 
  const maxAttempts = job.opts.attempts ?? 1
 
  if (job.attemptsMade >= maxAttempts) {
    console.error(`[DLQ] Job ${job.id} finally failed → transferring to DLQ`)
 
    await paymentDLQQueue.add(
      'failed-payment',
      {
        originalJobId: job.id ?? 'unknown',
        originalJobName: job.name,
        originalData: job.data,
        errorMessage: err instanceof Error ? err.message : String(err),
        attemptsMade: job.attemptsMade,
        failedAt: new Date().toISOString(),
      },
      {
        removeOnComplete: { age: 7 * 24 * 3600 },
        // If a worker is added to the DLQ for auto-reprocessing, prevents records from being auto-deleted even if that processing fails
        removeOnFail: false,
      }
    )
  }
})
 
paymentWorker.on('completed', (job) => {
  console.log(`[${job.id}] Payment completed: ${job.data.orderId}`)
})

DLQ Reprocessing Utility

A script for manually reprocessing DLQ jobs after the root cause has been resolved.

typescript

// scripts/reprocess-dlq.ts
import { Queue } from 'bullmq'
import { connection } from '../config/redis'
import type { DLQJobData } from '../queues/payment-dlq.queue'
 
const dlqQueue = new Queue<DLQJobData>('payments-dlq', { connection })
const paymentQueue = new Queue('payments', { connection })
 
async function listDLQJobs() {
  // Without a DLQ worker, jobs accumulate in 'waiting' state
  const jobs = await dlqQueue.getJobs(['waiting', 'delayed'])
  return jobs.map((job) => ({
    id: job.id,
    orderId: job.data.originalData.orderId,
    error: job.data.errorMessage,
    attempts: job.data.attemptsMade,
    failedAt: job.data.failedAt,
  }))
}
 
async function reprocessDLQJob(dlqJobId: string) {
  const dlqJob = await dlqQueue.getJob(dlqJobId)
  if (!dlqJob) throw new Error(`DLQ job ${dlqJobId} not found`)
 
  const newJob = await paymentQueue.add(
    dlqJob.data.originalJobName,
    dlqJob.data.originalData,
    {
      attempts: 3,
      backoff: { type: 'exponential', delay: 2000 },
      priority: 5,  // Reprocessed jobs get higher priority than regular jobs
    }
  )
 
  await dlqJob.remove()
 
  console.log(`Reprocessing complete: DLQ ${dlqJobId} → new job ${newJob.id}`)
  return newJob.id
}

Common Mistakes

Misconfiguring Redis `maxmemory-policy`

This is a silent and critical mistake. When Redis is shared with a cache and an eviction policy like allkeys-lru is active, job data being processed can be arbitrarily deleted when memory runs low.

Separate the Redis instance for BullMQ and always apply this setting.

bash

# redis.conf
maxmemory-policy noeviction

Not Setting `removeOnComplete` / `removeOnFail`

Completed jobs are not automatically removed. Specifying both count and age keeps memory usage predictable regardless of traffic patterns.

typescript

// count only: jobs from months ago can linger during low-traffic periods
// age only: memory can spike sharply during traffic surges
// Both together:
removeOnComplete: { count: 1000, age: 3600 }
removeOnFail: { count: 5000 }

Not Ensuring Idempotency in Processor Functions

typescript

// Example using Prisma
async function paymentProcessor(job: Job<PaymentJobData>) {
  const existing = await prisma.payment.findUnique({
    where: { idempotencyKey: job.data.idempotencyKey },
  })
 
  if (existing) {
    return existing  // Already processed payment: return previous result
  }
 
  return await prisma.payment.create({ data: { ...job.data } })
}

Overlooking the Concurrency Setting

The default concurrency for a Worker is 1. If throughput is lower than expected, check this setting first.

Diagram 3

For CPU-intensive work, tune based on core count; for I/O-intensive work, tune with load testing in the 20–50 range.

NestJS Integration

Using the @nestjs/bullmq package, you can modularize the same patterns with a decorator-based approach.

typescript

// notifications/notifications.module.ts
import { BullModule } from '@nestjs/bullmq'
import { Module } from '@nestjs/common'
import { NotificationProcessor } from './notification.processor'
import { NotificationService } from './notification.service'
 
@Module({
  imports: [
    BullModule.registerQueue({
      name: 'notifications',
      defaultJobOptions: {
        attempts: 3,
        backoff: { type: 'exponential', delay: 1000 },
        removeOnComplete: { count: 1000, age: 3600 },
        removeOnFail: { count: 5000 },
      },
    }),
  ],
  providers: [NotificationService, NotificationProcessor],
  exports: [NotificationService],
})
export class NotificationsModule {}

typescript

// notifications/notification.processor.ts
import { Processor, WorkerHost, OnWorkerEvent } from '@nestjs/bullmq'
import { Job } from 'bullmq'
import type { NotificationJobData } from './notification.types'
 
@Processor('notifications', { concurrency: 10 })
export class NotificationProcessor extends WorkerHost {
  async process(job: Job<NotificationJobData>): Promise<void> {
    switch (job.name) {
      case 'send-email':
        // sendEmail, sendPush are actual notification service clients — implementation omitted
        await this.sendEmail(job.data)
        break
      case 'send-push':
        await this.sendPush(job.data)
        break
      default:
        throw new Error(`Unknown job type: ${job.name}`)
    }
  }
 
  @OnWorkerEvent('failed')
  onFailed(job: Job<NotificationJobData> | undefined, err: Error) {
    if (!job) return  // job can be undefined in the failed event
    console.error(`Notification job failed [${job.id}]: ${err.message}`)
  }
}

Monitoring: Bull Board

typescript

// app.ts (Express-based)
import { createBullBoard } from '@bull-board/api'
import { BullMQAdapter } from '@bull-board/api/bullMQAdapter'
import { ExpressAdapter } from '@bull-board/express'
import { notificationQueue } from './queues/notification.queue'
import { paymentQueue } from './queues/payment.queue'
import { paymentDLQQueue } from './queues/payment-dlq.queue'
 
const serverAdapter = new ExpressAdapter()
serverAdapter.setBasePath('/admin/queues')
 
createBullBoard({
  queues: [
    new BullMQAdapter(notificationQueue),
    new BullMQAdapter(paymentQueue),
    new BullMQAdapter(paymentDLQQueue),
  ],
  serverAdapter,
})
 
app.use('/admin/queues', serverAdapter.getRouter())

Summary

Key patterns in summary:

Priority queue: Lower numbers are processed first. Can be changed at runtime with changePriority()
Retry backoff: The exponential strategy is most effective for external service outages. Custom functions can handle special cases like Rate Limits (but err instanceof Error type guard is required)
DLQ: Implemented manually using the failed event + attemptsMade >= attempts condition. Must be separated into its own queue to configure retention policies and monitoring independently

Three things to get right first in production:

Redis noeviction setting: Sharing an instance with an active eviction policy causes job data to silently disappear
Specify both count and age for removeOnComplete: Using either alone makes memory usage unpredictable depending on traffic patterns
Idempotency keys: With at-least-once guarantees, operations with side effects require DB-level deduplication beyond the queue layer

References

BullMQ Official Documentation — Complete guide to priority, retries, concurrency, and rate limiting
Prioritized Jobs | BullMQ Docs — Official priority queue documentation
Retrying Failing Jobs | BullMQ Docs — Official retry strategy documentation
BullMQ GitHub Release Notes — v5 series changelog
NestJS + BullMQ: Top 4 Use Cases & Quick Tutorial — NestJS integration practical examples
BullMQ Architecture for High Traffic — High-traffic architecture design

#BullMQ#TypeScript#Redis#Dead Letter Queue#우선순위 큐#백오프 재시도

Things to Consider When Choosing BullMQ

Core Concepts

BullMQ Architecture

Priority Queue

Retry Backoff

Dead Letter Queue

Production Implementation — Payment Processing Queue

Queue Definition

DLQ Queue Definition (Separated into Its Own File)

Worker and DLQ Transfer

DLQ Reprocessing Utility

Common Mistakes

Misconfiguring Redis maxmemory-policy

Not Setting removeOnComplete / removeOnFail

Not Ensuring Idempotency in Processor Functions

Overlooking the Concurrency Setting

NestJS Integration

Monitoring: Bull Board

Summary

References

Things to Consider When Choosing BullMQ

Core Concepts

BullMQ Architecture

Priority Queue

Retry Backoff

Dead Letter Queue

Production Implementation — Payment Processing Queue

Queue Definition

DLQ Queue Definition (Separated into Its Own File)

Worker and DLQ Transfer

DLQ Reprocessing Utility

Common Mistakes

Misconfiguring Redis maxmemory-policy

Not Setting removeOnComplete / removeOnFail

Not Ensuring Idempotency in Processor Functions

Overlooking the Concurrency Setting

NestJS Integration

Monitoring: Bull Board

Summary

References

Recommended Posts

Node.js Native TypeScript Execution Complete Guide

ElysiaJS 1.0 in Practice: Building a Type-Safe REST API with Bun's Native Framework

Bun 1.2 Built-in SQLite for a Zero-Dependency Local-First Backend

BullMQ 5 + Redis로 Node.js 백그라운드 작업 큐 운영하기

Elysia.js + Bun으로 서버-클라이언트 타입을 한 번에 묶는 법

A Rust API Server That Catches SQL Errors at Build Time — Axum + SQLx Type-Safe Patterns Explained

Misconfiguring Redis `maxmemory-policy`

Not Setting `removeOnComplete` / `removeOnFail`

Misconfiguring Redis `maxmemory-policy`

Not Setting `removeOnComplete` / `removeOnFail`