Delivery Pipeline

The notification delivery pipeline transforms domain events into delivered notifications across three channels. It separates planning (synchronous, in the app) from execution (asynchronous, via workers).

Source: src/lib/notifications/notify.ts, src/lib/notifications/delivery.ts

Pipeline Overview

Phase 1: Planning

When a domain event triggers a notification (via integration publishers), planNotification() orchestrates the planning phase:

1. Load recipient profile -- Fetches user preferences, timezone, active push subscriptions
2. Build preference snapshot -- Resolves effective preferences including mute state, quiet hours, collaboration mode
3. Generate keys -- Creates dedupe key (FNV hash), batch key, and group key
4. Decide each channel -- Independently evaluates in-app, email, and push using the type's registered policy

Channel Decision Actions

Each channel decision results in one of:

Action	Status	When
send_now	pending	Immediate delivery, no scheduling needed
schedule	scheduled	Deferred (digest, quiet hours, collaboration delay)
batch	scheduled	Grouped with related notifications (batch key set)
skip	skipped	Channel disabled, muted, or suppressed

Deduplication

After planning, notifyWithStore() checks for an existing notification with the same dedupe key within the 24-hour window. If found, all deliveries are marked as skipped with reason deduplicated.

The dedupe key is a stable FNV hash of: recipient_id + actor_id + context + title + body + payload.

Database Writes

Planning results in:

• 1 notification record in notifications (content, metadata, grouping keys)
• 1-3 delivery records in notification_deliveries (one per non-skipped channel)

If delivery insertion fails, the notification record is rolled back.

Phase 2: Scheduling

The notification-scheduler worker runs every minute and:

1. Claims due deliveries using SELECT ... FOR UPDATE SKIP LOCKED:
   • status IN ('pending', 'scheduled') AND scheduled_for <= NOW()
   • status = 'failed' AND next_retry_at <= NOW() AND attempts < max_attempts
2. Sorts by priority (immediate > normal > low), then by due time
3. Updates status to queued with queued_at timestamp
4. Enqueues messages to the notification-deliveries Cloudflare Queue
5. Cleans up expired tracking tokens (30+ days) and old dead letters (90+ days)

Queue Message Schema

interface NotificationDeliveryQueueMessage {
	version: 1;
	type: 'notification_delivery.execute';
	deliveryId: string;
	notificationId: string;
	channel: 'email' | 'in_app' | 'push';
	priority: 'low' | 'normal' | 'immediate';
	reason: 'due' | 'retry';
	queuedAt: string; // ISO timestamp
}

Scheduler Configuration

Setting	Default	Purpose
SCHEDULER_PAGE_SIZE	50	Records per page claim
SCHEDULER_MAX_RECORDS_PER_RUN	200	Total records per cron invocation
NOTIFICATION_CLEANUP_BATCH_SIZE	100	Cleanup batch size
NOTIFICATION_TRACKING_TOKEN_RETENTION_DAYS	30	Token expiry
NOTIFICATION_DEAD_LETTER_RETENTION_DAYS	90	Dead letter retention

Phase 3: Execution

The notification-executor worker consumes queue messages in batches of up to 25:

Pre-Delivery Checks

Before executing, the worker runs several checks that may skip or reschedule:

1. Push subscription check -- If channel is push and no active subscriptions exist, skip with reason push_subscription_revoked
2. Quiet hours re-evaluation -- Resolves recipient preferences fresh; if currently in quiet hours, reschedule to quietHours.endsAt
3. Read-state sensitivity -- For chat/mention emails: if the notification has been read (user saw it in-app), skip the email
4. Hourly email cap -- For chat notifications with hourly_cap_key: if an email was already sent in this hourly bucket, skip

Channel Execution

Channel	Executor	Provider	Result
In-App	createInAppChannelExecutor()	Supabase Realtime	Immediate delivered
Email	createEmailChannelExecutor()	Resend API	sent with tracking tokens
Push	createPushChannelExecutor()	Web Push API (VAPID)	sent per subscription

Result Handling

Retry Strategy

Retries use escalating delays:

Attempt	Delay
1	5 minutes
2	15 minutes
3	60 minutes
4+	180 minutes

Error Classification

Error Type	Class	Behaviour
Network timeout	NotificationTransientDeliveryError	Retry with backoff
Rate limit (429)	NotificationTransientDeliveryError	Retry with backoff
Server error (5xx)	NotificationTransientDeliveryError	Retry with backoff
Invalid email	NotificationPermanentDeliveryError	Dead-letter immediately
Bad configuration	NotificationPermanentDeliveryError	Dead-letter immediately
Unsupported type	NotificationPermanentDeliveryError	Dead-letter immediately

Delivery Status Lifecycle

Store Abstraction

All database operations are abstracted behind store interfaces, enabling testability:

• NotificationStore -- Planning operations (insert notification/deliveries, dedup lookup)
• NotificationSchedulerStore -- Scheduler operations (claim pages, enqueue, cleanup)
• NotificationDeliveryExecutorStore -- Executor operations (claim, mark sent/failed/skipped, retry)
• NotificationChannelExecutor -- Per-channel delivery interface

Related Documentation

• Notification System Overview -- Architecture and notification types
• Preferences & Policies -- How delivery decisions are made
• Notification Scheduler Worker -- Cron worker details
• Notification Executor Worker -- Queue worker details

Push latency budget

Two poll boundaries sit between a notification being created and a push arriving, and both were measured in production:

Hop	Source	Cost
created_at to queued_at	notification-scheduler cron, * * * * *	0 to 60s
queued_at to sent_at	executor queue max_batch_timeout	was a flat ~32s at 30, now ~1s

max_batch_timeout is deliberately 1 on the notification-deliveries consumer: latency matters far more than batching efficiency for a notification, and max_batch_size = 25 still batches whatever is already waiting.

The remaining minute is the scheduler poll. Removing it means producing to the delivery queue directly at notify time for send_now deliveries, claiming the row atomically first so the scheduler cannot enqueue it a second time. Not done yet.