Obsah článku / Contents
Priority Queue: Keď FIFO nestačí a potrebuješ VIP pruh

Priority Queue: Keď FIFO nestačí a potrebuješ VIP pruh

"Všetky úlohy sú rovnaké, ale niektoré sú rovnejšie!" 🎯

Prolog: The Problem

python
# Piatok 14:00
boss = "Potrebujem queue systém!"
you = "OK, FIFO queue?"
boss = "Nie! Niektoré úlohy sú dôležitejšie!"
you = "OK, priority queue?"
boss = "Áno! A keď sa nespracujú včas, repriotizuj ich!"
you = "...wat?"
boss = "A potrebujem to do pondelka!"
you = "😰"

# Googling intensifies...

Real-world scenár:

E-commerce systém:
- Email notifikácie (LOW priority)
- Generovanie faktúr (MEDIUM priority)  
- Platobné transakcie (HIGH priority)
- Fraud detection (CRITICAL priority)

Problem:
- FIFO: Všetko rovnako → fraud čaká za 1000 emailmi
- Priority: OK, ale čo keď stuck?
- TTL: Timeout, ale čo potom?

Solution needed:
✅ Priority-based processing
✅ TTL (time to live)
✅ Auto re-prioritization
✅ Dead letter handling

Poďme na to!


Kapitola 1: FIFO Fail

Classic Queue Problem

python
# Klasická FIFO queue
queue = [
    {"type": "email", "data": "Welcome email 1"},
    {"type": "email", "data": "Welcome email 2"},
    {"type": "email", "data": "Welcome email 3"},
    # ... 997 more emails ...
    {"type": "fraud", "data": "Suspicious transaction $10,000"}  # STUCK!
]

# Fraud detection čaká na 1000 emailov
# Meantime: Customer's credit card already drained
# Customer: 😡
# Boss: 😡😡
# You: 😰

Why FIFO fails:

Time: 09:00 - 1000 emails queued
Time: 09:01 - FRAUD detected, added to queue
Time: 09:02 - Still processing emails...
Time: 09:10 - Still processing emails...
Time: 09:30 - Finally fraud check! (30 min delay)
Time: 09:31 - Customer account already compromised

Result: ❌ FAIL

Priority to the rescue?

python
# Naive priority implementation
priority_queue = {
    "CRITICAL": [],
    "HIGH": [],
    "MEDIUM": [],
    "LOW": []
}

# Process CRITICAL first, then HIGH, etc.
while True:
    if priority_queue["CRITICAL"]:
        process(priority_queue["CRITICAL"].pop(0))
    elif priority_queue["HIGH"]:
        process(priority_queue["HIGH"].pop(0))
    # ...

Problems:

1. Starvation
   - LOW priority never processed
   - Emails pile up forever
   
2. No TTL
   - Task stuck? No timeout!
   - Hangs forever
   
3. No re-prioritization  
   - OLD task still LOW priority
   - Even after 2 days!
   
4. Manual implementation
   - Error-prone
   - Hard to maintain
   - Reinventing wheel

Potrebujeme profesionálne riešenie!


Kapitola 2: RabbitMQ Solution

Why RabbitMQ?

python
rabbitmq_features = {
    "priority_queues": "Built-in! ✅",
    "ttl": "Per-message TTL ✅",
    "dead_letter_exchange": "Auto re-queue ✅",
    "message_durability": "Persistent ✅",
    "acknowledgments": "Reliable ✅",
    "clustering": "High availability ✅"
}

verdict = "Perfect match!" 🎯

Architecture

┌─────────────┐
│  Producer   │
└──────┬──────┘
       │ publish(msg, priority=9)
       ↓
┌─────────────────────────────┐
│    Exchange (topic)         │
└──────────┬──────────────────┘
           │ route by priority
           ↓
    ┌──────────────┐
    │ Priority Queue│ priority=10 (max)
    │   x-max-priority=10
    │   x-message-ttl=30000
    │   x-dead-letter-exchange=dlx
    └──────┬───────┘
           │ TTL expired?
           ↓ YES
    ┌──────────────┐
    │ Dead Letter  │
    │  Exchange    │
    └──────┬───────┘
           │ re-route with higher priority
           ↓
    ┌──────────────┐
    │ Retry Queue  │ priority increased!
    └──────┬───────┘
           │
           ↓
    ┌──────────────┐
    │  Consumer    │
    └──────────────┘

Kapitola 3: Implementation

Setup RabbitMQ (Docker)

bash
# docker-compose.yml
version: '3.8'
services:
  rabbitmq:
    image: rabbitmq:3-management
    ports:
      - "5672:5672"    # AMQP
      - "15672:15672"  # Management UI
    environment:
      RABBITMQ_DEFAULT_USER: admin
      RABBITMQ_DEFAULT_PASS: admin123

# Start
docker-compose up -d

# Management UI: http://localhost:15672
# Login: admin / admin123

Python Implementation

python
# requirements.txt
pika==1.3.2

# config.py
import pika

class RabbitMQConfig:
    HOST = 'localhost'
    PORT = 5672
    USER = 'admin'
    PASSWORD = 'admin123'
    
    # Queue settings
    MAIN_QUEUE = 'task_queue'
    DLX_EXCHANGE = 'dlx_exchange'
    RETRY_QUEUE = 'retry_queue'
    
    # Priorities
    PRIORITY_CRITICAL = 10
    PRIORITY_HIGH = 7
    PRIORITY_MEDIUM = 5
    PRIORITY_LOW = 2
    
    # TTL (milliseconds)
    TTL_CRITICAL = 5000   # 5 seconds
    TTL_HIGH = 15000      # 15 seconds
    TTL_MEDIUM = 30000    # 30 seconds
    TTL_LOW = 60000       # 60 seconds

def get_connection():
    credentials = pika.PlainCredentials(
        RabbitMQConfig.USER,
        RabbitMQConfig.PASSWORD
    )
    parameters = pika.ConnectionParameters(
        host=RabbitMQConfig.HOST,
        port=RabbitMQConfig.PORT,
        credentials=credentials
    )
    return pika.BlockingConnection(parameters)

Setup Queues

python
# setup_queues.py

import pika
from config import get_connection, RabbitMQConfig

def setup_queues():
    connection = get_connection()
    channel = connection.channel()
    
    # 1. Dead Letter Exchange (for TTL expired messages)
    channel.exchange_declare(
        exchange=RabbitMQConfig.DLX_EXCHANGE,
        exchange_type='topic',
        durable=True
    )
    
    # 2. Main Priority Queue
    channel.queue_declare(
        queue=RabbitMQConfig.MAIN_QUEUE,
        durable=True,
        arguments={
            'x-max-priority': 10,  # Max priority level
            'x-dead-letter-exchange': RabbitMQConfig.DLX_EXCHANGE,
            'x-dead-letter-routing-key': 'retry'
        }
    )
    
    # 3. Retry Queue (for re-prioritization)
    channel.queue_declare(
        queue=RabbitMQConfig.RETRY_QUEUE,
        durable=True,
        arguments={
            'x-max-priority': 10,
            'x-message-ttl': 5000  # Wait 5s before retry
        }
    )
    
    # 4. Bind queues
    channel.queue_bind(
        exchange=RabbitMQConfig.DLX_EXCHANGE,
        queue=RabbitMQConfig.RETRY_QUEUE,
        routing_key='retry'
    )
    
    print("✅ Queues setup complete!")
    connection.close()

if __name__ == '__main__':
    setup_queues()

Producer (Send Messages)

python
# producer.py

import pika
import json
from datetime import datetime
from config import get_connection, RabbitMQConfig

class TaskProducer:
    def __init__(self):
        self.connection = get_connection()
        self.channel = self.connection.channel()
    
    def send_task(self, task_type, data, priority=5):
        """
        Send task to queue with priority
        
        Args:
            task_type: Type of task (email, invoice, fraud, etc.)
            data: Task data (dict)
            priority: 1-10 (10 = highest)
        """
        # Determine TTL based on priority
        ttl_map = {
            10: RabbitMQConfig.TTL_CRITICAL,
            7: RabbitMQConfig.TTL_HIGH,
            5: RabbitMQConfig.TTL_MEDIUM,
            2: RabbitMQConfig.TTL_LOW
        }
        ttl = ttl_map.get(priority, RabbitMQConfig.TTL_MEDIUM)
        
        message = {
            'task_type': task_type,
            'data': data,
            'priority': priority,
            'created_at': datetime.now().isoformat(),
            'retry_count': 0
        }
        
        # Publish with priority and TTL
        self.channel.basic_publish(
            exchange='',
            routing_key=RabbitMQConfig.MAIN_QUEUE,
            body=json.dumps(message),
            properties=pika.BasicProperties(
                priority=priority,
                expiration=str(ttl),  # TTL in milliseconds
                delivery_mode=2,      # Persistent
                content_type='application/json'
            )
        )
        
        print(f"✅ Sent: {task_type} (priority={priority}, ttl={ttl}ms)")
    
    def close(self):
        self.connection.close()

# Usage example
if __name__ == '__main__':
    producer = TaskProducer()
    
    # Send tasks with different priorities
    producer.send_task('email', {'to': 'user@example.com'}, priority=2)
    producer.send_task('invoice', {'order_id': 123}, priority=5)
    producer.send_task('payment', {'amount': 99.99}, priority=7)
    producer.send_task('fraud_check', {'transaction_id': 456}, priority=10)
    
    producer.close()

Consumer (Process Messages)

python
# consumer.py

import pika
import json
import time
from config import get_connection, RabbitMQConfig

class TaskConsumer:
    def __init__(self):
        self.connection = get_connection()
        self.channel = self.connection.channel()
        
        # Set QoS (Quality of Service)
        # Process only 1 message at a time (fair dispatch)
        self.channel.basic_qos(prefetch_count=1)
    
    def process_task(self, task_type, data, retry_count):
        """Process task based on type"""
        print(f"\n📌 Processing: {task_type} (retry: {retry_count})")
        
        try:
            if task_type == 'email':
                self.send_email(data)
            elif task_type == 'invoice':
                self.generate_invoice(data)
            elif task_type == 'payment':
                self.process_payment(data)
            elif task_type == 'fraud_check':
                self.check_fraud(data)
            else:
                print(f"❌ Unknown task type: {task_type}")
                return False
            
            return True
        
        except Exception as e:
            print(f"❌ Error processing {task_type}: {e}")
            return False
    
    def send_email(self, data):
        time.sleep(1)  # Simulate work
        print(f"📧 Email sent to {data.get('to')}")
    
    def generate_invoice(self, data):
        time.sleep(2)  # Simulate work
        print(f"📄 Invoice generated for order {data.get('order_id')}")
    
    def process_payment(self, data):
        time.sleep(3)  # Simulate work
        print(f"💳 Payment processed: ${data.get('amount')}")
    
    def check_fraud(self, data):
        time.sleep(1)  # Simulate work
        print(f"🔍 Fraud check for transaction {data.get('transaction_id')}")
    
    def callback(self, ch, method, properties, body):
        """Callback for message processing"""
        message = json.loads(body)
        
        task_type = message['task_type']
        data = message['data']
        retry_count = message.get('retry_count', 0)
        priority = message.get('priority', 5)
        
        print(f"\n{'='*50}")
        print(f"📥 Received: {task_type}")
        print(f"   Priority: {priority}")
        print(f"   Retry: {retry_count}")
        print(f"{'='*50}")
        
        # Process task
        success = self.process_task(task_type, data, retry_count)
        
        if success:
            # ACK - message processed successfully
            ch.basic_ack(delivery_tag=method.delivery_tag)
            print(f"✅ ACK: {task_type}")
        else:
            # NACK - requeue message
            ch.basic_nack(
                delivery_tag=method.delivery_tag,
                requeue=False  # Send to DLX instead
            )
            print(f"❌ NACK: {task_type} (sending to DLX)")
    
    def start_consuming(self):
        """Start consuming messages"""
        print("🚀 Consumer started. Waiting for messages...")
        print("   Press Ctrl+C to stop")
        
        self.channel.basic_consume(
            queue=RabbitMQConfig.MAIN_QUEUE,
            on_message_callback=self.callback
        )
        
        try:
            self.channel.start_consuming()
        except KeyboardInterrupt:
            print("\n⛔ Consumer stopped")
            self.channel.stop_consuming()
        finally:
            self.connection.close()

if __name__ == '__main__':
    consumer = TaskConsumer()
    consumer.start_consuming()

Retry Handler (Re-prioritization)

python
# retry_handler.py

import pika
import json
from config import get_connection, RabbitMQConfig

class RetryHandler:
    """
    Handle messages from retry queue
    Re-prioritize and send back to main queue
    """
    
    def __init__(self):
        self.connection = get_connection()
        self.channel = self.connection.channel()
        self.channel.basic_qos(prefetch_count=1)
    
    def callback(self, ch, method, properties, body):
        message = json.loads(body)
        
        # Increase priority for retry
        old_priority = message.get('priority', 5)
        new_priority = min(old_priority + 2, 10)  # Max 10
        
        retry_count = message.get('retry_count', 0) + 1
        message['retry_count'] = retry_count
        message['priority'] = new_priority
        
        print(f"\n♻️  Retrying task: {message['task_type']}")
        print(f"   Old priority: {old_priority}")
        print(f"   New priority: {new_priority}")
        print(f"   Retry count: {retry_count}")
        
        # Check max retries
        if retry_count > 5:
            print(f"⛔ Max retries exceeded. Moving to dead letter.")
            # Move to permanent dead letter queue
            ch.basic_ack(delivery_tag=method.delivery_tag)
            # In production: store in DB or alert
            return
        
        # Re-publish to main queue with higher priority
        self.channel.basic_publish(
            exchange='',
            routing_key=RabbitMQConfig.MAIN_QUEUE,
            body=json.dumps(message),
            properties=pika.BasicProperties(
                priority=new_priority,
                delivery_mode=2,
                content_type='application/json'
            )
        )
        
        ch.basic_ack(delivery_tag=method.delivery_tag)
        print(f"✅ Re-queued with priority {new_priority}")
    
    def start(self):
        print("🔄 Retry handler started...")
        
        self.channel.basic_consume(
            queue=RabbitMQConfig.RETRY_QUEUE,
            on_message_callback=self.callback
        )
        
        try:
            self.channel.start_consuming()
        except KeyboardInterrupt:
            print("\n⛔ Retry handler stopped")
            self.channel.stop_consuming()
        finally:
            self.connection.close()

if __name__ == '__main__':
    handler = RetryHandler()
    handler.start()

Kapitola 4: Testing

Test Scenario

python
# test_priority.py

import time
from producer import TaskProducer

def test_priority_processing():
    """
    Test that higher priority tasks are processed first
    """
    producer = TaskProducer()
    
    print("📤 Sending tasks in wrong order...")
    
    # Send in wrong order (LOW first)
    producer.send_task('email', {'to': 'user1@test.com'}, priority=2)
    time.sleep(0.1)
    
    producer.send_task('email', {'to': 'user2@test.com'}, priority=2)
    time.sleep(0.1)
    
    producer.send_task('invoice', {'order_id': 123}, priority=5)
    time.sleep(0.1)
    
    # CRITICAL task last!
    producer.send_task('fraud_check', {'transaction_id': 999}, priority=10)
    
    producer.close()
    
    print("\n✅ Tasks sent. Check consumer output.")
    print("   Expected: Fraud check processed FIRST!")

def test_ttl_expiration():
    """
    Test TTL expiration and re-prioritization
    """
    producer = TaskProducer()
    
    print("📤 Sending task with short TTL...")
    
    # Send task with priority 5 (TTL = 30s)
    producer.send_task('slow_task', {'data': 'test'}, priority=5)
    
    producer.close()
    
    print("\n⏱️  If not processed in 30s:")
    print("   1. Message expires (TTL)")
    print("   2. Sent to DLX")
    print("   3. Re-queued with higher priority")
    print("   4. Processed with priority 7")

if __name__ == '__main__':
    print("=== Priority Test ===")
    test_priority_processing()
    
    print("\n\n=== TTL Test ===")
    test_ttl_expiration()

Running the System

bash
# Terminal 1: Start consumer
python consumer.py

# Terminal 2: Start retry handler
python retry_handler.py

# Terminal 3: Send test messages
python test_priority.py

# Expected output (Terminal 1):
# ====================================================
# 📥 Received: fraud_check       <- PROCESSED FIRST!
#    Priority: 10
# ====================================================
# 🔍 Fraud check for transaction 999
# ✅ ACK: fraud_check
#
# ====================================================
# 📥 Received: invoice            <- Then medium
#    Priority: 5
# ====================================================
# ...

Kapitola 5: Porovnanie s alternatívami

RabbitMQ vs Redis vs Kafka

python
comparison = {
    "RabbitMQ": {
        "priority_queues": "✅ Native support",
        "ttl": "✅ Per-message TTL",
        "dlx": "✅ Dead letter exchange",
        "re_prioritization": "✅ Easy with DLX",
        "reliability": "✅✅ AMQP protocol",
        "complexity": "Medium",
        "best_for": "Task queues, job processing"
    },
    "Redis": {
        "priority_queues": "🔶 Manual (sorted sets)",
        "ttl": "✅ EXPIRE command",
        "dlx": "❌ Manual implementation",
        "re_prioritization": "🔶 Manual (ZADD)",
        "reliability": "🔶 Optional persistence",
        "complexity": "Low",
        "best_for": "Caching, simple queues"
    },
    "Kafka": {
        "priority_queues": "❌ No native support",
        "ttl": "❌ No per-message TTL",
        "dlx": "❌ Manual implementation",
        "re_prioritization": "❌ Complex workaround",
        "reliability": "✅✅ Distributed log",
        "complexity": "High",
        "best_for": "Event streaming, logs"
    }
}

Redis Implementation (for comparison)

python
# Redis approach (more manual)
import redis
import json
import time

class RedisPriorityQueue:
    def __init__(self):
        self.redis = redis.Redis(host='localhost', port=6379)
        self.queue_key = 'priority_queue'
    
    def add_task(self, task, priority):
        """
        Add task to sorted set
        Score = -priority (higher priority = lower score = processed first)
        """
        task_data = json.dumps(task)
        # Use negative priority so higher priority = lower score
        self.redis.zadd(self.queue_key, {task_data: -priority})
    
    def get_task(self):
        """
        Get highest priority task
        """
        # Get task with lowest score (highest priority)
        result = self.redis.zpopmin(self.queue_key)
        if result:
            task_data, score = result[0]
            return json.loads(task_data)
        return None
    
    def set_ttl(self, task_id, ttl_seconds):
        """
        Manual TTL implementation
        """
        key = f"task:{task_id}"
        self.redis.expire(key, ttl_seconds)
    
    # Problem: No automatic re-prioritization!
    # Need to implement manually with Redis Keyspace Notifications
    # or separate worker checking expired tasks

# Usage
queue = RedisPriorityQueue()
queue.add_task({'type': 'email'}, priority=2)
queue.add_task({'type': 'fraud'}, priority=10)

task = queue.get_task()  # Gets fraud (priority 10)

Redis problémy:

❌ No automatic DLX equivalent
❌ Manual TTL handling
❌ No automatic re-prioritization
❌ Need custom logic for everything
✅ Simple, fast
✅ Good for basic use cases

Kafka Implementation (not suitable)

python
# Kafka is NOT designed for priority queues!

# Kafka approach (workaround, not recommended)
class KafkaPriorityQueue:
    """
    Kafka workaround: Multiple topics per priority
    NOT recommended for task queues!
    """
    
    def __init__(self):
        self.producer = KafkaProducer()
        self.topics = {
            'critical': 'tasks-priority-10',
            'high': 'tasks-priority-7',
            'medium': 'tasks-priority-5',
            'low': 'tasks-priority-2'
        }
    
    def send_task(self, task, priority):
        # Send to different topic based on priority
        if priority >= 9:
            topic = self.topics['critical']
        elif priority >= 7:
            topic = self.topics['high']
        # ...
        
        self.producer.send(topic, task)
    
    # Problem:
    # - Need multiple consumers
    # - No cross-topic priority
    # - No TTL per message
    # - Complex orchestration

# Verdict: Use Kafka for event streaming, NOT task queues!

Kafka problémy:

❌ No priority queues (by design!)
❌ No per-message TTL
❌ Workarounds are complex
❌ Over-engineered for simple tasks
✅ Excellent for event streaming
✅ High throughput
✅ Perfect for logs, analytics

Kapitola 6: Production Considerations

Monitoring

python
# monitoring.py

from prometheus_client import Counter, Histogram, Gauge
import pika

# Metrics
tasks_processed = Counter(
    'tasks_processed_total',
    'Total tasks processed',
    ['task_type', 'priority']
)

task_duration = Histogram(
    'task_duration_seconds',
    'Task processing duration',
    ['task_type']
)

queue_size = Gauge(
    'queue_size',
    'Current queue size',
    ['queue_name']
)

retry_count = Counter(
    'task_retries_total',
    'Total task retries',
    ['task_type']
)

def get_queue_size(queue_name):
    """Get current queue size from RabbitMQ"""
    connection = get_connection()
    channel = connection.channel()
    
    queue = channel.queue_declare(
        queue=queue_name,
        passive=True  # Don't create, just check
    )
    
    size = queue.method.message_count
    connection.close()
    
    return size

# Update metrics
queue_size.labels(queue_name='task_queue').set(
    get_queue_size('task_queue')
)

High Availability

python
# ha_config.py

# RabbitMQ cluster configuration
rabbitmq_nodes = [
    {'host': 'rabbitmq-1', 'port': 5672},
    {'host': 'rabbitmq-2', 'port': 5672},
    {'host': 'rabbitmq-3', 'port': 5672}
]

def get_ha_connection():
    """Connect to RabbitMQ cluster"""
    import random
    
    # Random node selection (or use load balancer)
    node = random.choice(rabbitmq_nodes)
    
    credentials = pika.PlainCredentials('admin', 'admin123')
    parameters = pika.ConnectionParameters(
        host=node['host'],
        port=node['port'],
        credentials=credentials,
        heartbeat=60,
        blocked_connection_timeout=300
    )
    
    return pika.BlockingConnection(parameters)

# Queue policy for HA
# (Run on RabbitMQ server)
"""
rabbitmqctl set_policy ha-all "^task" \
  '{"ha-mode":"all","ha-sync-mode":"automatic"}'

# This ensures queue is mirrored across all nodes
"""

Error Handling

python
# error_handler.py

import logging
from datetime import datetime

class TaskErrorHandler:
    def __init__(self):
        self.logger = logging.getLogger(__name__)
        self.error_queue = 'errors'
    
    def handle_error(self, task, error, retry_count):
        """
        Handle task processing error
        """
        error_log = {
            'task': task,
            'error': str(error),
            'retry_count': retry_count,
            'timestamp': datetime.now().isoformat()
        }
        
        # Log error
        self.logger.error(f"Task error: {error_log}")
        
        # Store in error queue for analysis
        self.store_error(error_log)
        
        # Alert if critical task failed multiple times
        if task.get('priority') >= 9 and retry_count > 3:
            self.send_alert(task, error, retry_count)
    
    def store_error(self, error_log):
        """Store error in database or error queue"""
        # In production: store in DB, send to error tracking service
        pass
    
    def send_alert(self, task, error, retry_count):
        """Send alert for critical task failures"""
        # In production: PagerDuty, Slack, email
        print(f"🚨 ALERT: Critical task failed {retry_count} times!")
        print(f"   Task: {task}")
        print(f"   Error: {error}")

Kapitola 7: Real-World Use Cases

Use Case 1: E-commerce Order Processing

python
# e-commerce example

class OrderProcessor:
    def send_to_queue(self, order):
        producer = TaskProducer()
        
        # Determine priority based on order type
        if order['total'] > 1000:
            priority = 10  # VIP order
        elif order['express_shipping']:
            priority = 8   # Express
        elif order['total'] > 100:
            priority = 5   # Regular
        else:
            priority = 3   # Low value
        
        # Send to queue
        producer.send_task(
            task_type='process_order',
            data={
                'order_id': order['id'],
                'total': order['total'],
                'items': order['items']
            },
            priority=priority
        )
        
        producer.close()

# Result:
# - VIP orders (>$1000) processed first
# - Express shipping processed quickly
# - Regular orders processed normally
# - Low-value orders processed when queue is free

Use Case 2: Notification System

python
# notification system

class NotificationService:
    PRIORITIES = {
        'security_alert': 10,      # Highest
        'password_reset': 9,
        'order_confirmation': 7,
        'shipping_update': 5,
        'marketing_email': 2,      # Lowest
        'newsletter': 1
    }
    
    def send_notification(self, notification_type, recipient, content):
        producer = TaskProducer()
        
        priority = self.PRIORITIES.get(notification_type, 5)
        
        producer.send_task(
            task_type='send_notification',
            data={
                'type': notification_type,
                'recipient': recipient,
                'content': content
            },
            priority=priority
        )
        
        producer.close()

# Usage
service = NotificationService()

# These are processed in priority order:
service.send_notification('newsletter', 'user@test.com', '...')  # Priority 1
service.send_notification('security_alert', 'user@test.com', '...')  # Priority 10 (FIRST!)
service.send_notification('marketing_email', 'user@test.com', '...')  # Priority 2

Use Case 3: Media Processing Pipeline

python
# media processing

class MediaProcessor:
    def process_media(self, media_id, media_type, user_tier):
        producer = TaskProducer()
        
        # Priority based on user tier and media type
        priority_map = {
            ('premium', 'video'): 10,
            ('premium', 'image'): 8,
            ('free', 'video'): 5,
            ('free', 'image'): 3
        }
        
        priority = priority_map.get((user_tier, media_type), 5)
        
        producer.send_task(
            task_type='process_media',
            data={
                'media_id': media_id,
                'media_type': media_type,
                'user_tier': user_tier
            },
            priority=priority
        )
        
        producer.close()

# Result:
# - Premium users' videos processed first
# - Free users' images processed last
# - Fair queue for everyone

Záver: Why RabbitMQ Wins

python
requirements = {
    "priority_queues": "✅ Native",
    "ttl_per_message": "✅ Built-in",
    "auto_reprioritization": "✅ DLX + logic",
    "reliability": "✅ AMQP",
    "monitoring": "✅ Management UI",
    "clustering": "✅ HA support"
}

alternatives = {
    "redis": "Manual implementation needed",
    "kafka": "Not designed for task queues"
}

verdict = """
RabbitMQ je perfektný pre:
✅ Task queues s prioritami
✅ TTL a automatic retry
✅ Dead letter handling
✅ Re-prioritization
✅ Production-ready

Redis je lepší pre:
- Simple queues
- Caching
- Pub/Sub

Kafka je lepší pre:
- Event streaming
- Log aggregation
- Analytics pipeline

Choose the right tool! 🎯
"""

Final architecture:

Producer → RabbitMQ Priority Queue
              ↓
           Consumer
              ↓
          (Success?)
           ↙     ↘
         YES      NO
          ↓       ↓
         ACK     DLX
                  ↓
            Retry Queue
                  ↓
          Higher Priority
                  ↓
         Back to Main Queue

Výhody:

✅ FIFO pre rovnakú prioritu
✅ Priorita pre dôležité úlohy
✅ TTL pre timeout detection
✅ Automatic re-queue s vyššou prioritou
✅ Dead letter pre final failures
✅ Production-ready out of the box
✅ No reinventing wheel

Článok napísal developer ktorý potreboval priority queue a našiel RabbitMQ. Sometimes you don't need to build it yourself. Use battle-tested solutions. 🐰

P.S.: FIFO je OK, ale nie pre všetko. VIP pruh existuje v reálnom svete. Prečo nie v queue? 😊

P.P.S.: Redis a Kafka sú skvelé nástroje, ale nie pre tento use case. Right tool for the job! 🔧

Všetky články