Back to Blog
Backend
Building Real-Time Notification Systems
December 10, 2024
2 min read
Building Real-Time Notification Systems
At Skippa Trading, I built the notification infrastructure that delivers trade alerts to users. In trading, speed matters—every millisecond counts. Here's how we approached it.
The Requirements
- Multi-channel delivery: Discord, push notifications (OneSignal), and WebSocket
- Low latency: Notifications need to arrive as fast as possible
- High reliability: Traders depend on these alerts for their decisions
- Graceful degradation: If one channel fails, others should still work
Architecture Overview
[Event Source]
↓
[Message Queue (Redis)]
↓
[Notification Worker Pool]
↓
┌────────────┬────────────┬────────────┐
│ Discord │ OneSignal │ WebSocket │
└────────────┴────────────┴────────────┘
Why Redis?
Redis Pub/Sub gives us:
- Sub-millisecond message delivery
- Built-in support for multiple subscribers
- Simple to operate and monitor
- Excellent Node.js client support
Key Optimizations
1. Connection Pooling
Maintain persistent connections to external services instead of connecting per-request.
// Bad: New connection per notification
async function sendDiscordAlert(message: string) {
const client = new DiscordClient(token);
await client.connect();
await client.send(message);
await client.disconnect();
}
// Good: Reuse connection
const discordClient = new DiscordClient(token);
await discordClient.connect();
async function sendDiscordAlert(message: string) {
await discordClient.send(message);
}2. Batch Processing
When possible, batch notifications instead of sending them one by one.
3. Priority Queues
Not all notifications are equal. Critical trading alerts get priority over general updates.
interface Notification {
priority: 'critical' | 'high' | 'normal';
channel: string;
payload: unknown;
}Monitoring
You can't improve what you don't measure. We track:
- Delivery latency per channel
- Failure rates by channel and error type
- Queue depth to catch backlogs early
- End-to-end time from event to delivery
Lessons Learned
- Test failure modes - What happens when Discord is down? When Redis restarts?
- Set timeouts everywhere - External services will hang. Don't let them block your system.
- Log correlation IDs - When debugging, you need to trace a notification through every step.
Building a notification system? Reach out if you want to discuss architecture.