Campaign Processing

📦 "Producer – Database – Consumer" Architecture

---

🧭 Overview

This approach organizes asynchronous task execution with reliable synchronization, execution guarantees, and fault tolerance. It is based on the following pattern:

1. The Producer creates a task entity (e.g., CampaignExecution) and saves it to the database with the status NEW.

2. The Consumer periodically scans the database, finds such tasks, and executes them.

The database acts as a task queue, ensuring:

• Atomic creation and selection of tasks

• Retry support in case of failure

• Resilience in clustered environments and during restarts

---

🏭 Components in the Context of Campaigns

1. CampaignExecutionProducer

Purpose: Periodically checks if a new CampaignExecution should be created for active campaigns.

Technical Features:

• Uses @Scheduled + Shedlock to:

  • Ensure single-instance execution in distributed deployments

  • Prevent concurrent access

• Creates a CampaignExecution with status NEW and stores it in the DB.

📌 Note: The producer does not execute the campaign — it only prepares the task.

---

2. Database

Acts as the central component:

• Stores the list of all CampaignExecution entities and their statuses

• Supports getSync(id) for safe transactional access

• Enables filtering by status NEW to identify tasks for execution

Advantages of DB-backed queueing:

• Reliability: Data persists even during application crashes

• Transparency: Task status is visible in the UI and easy to debug

• Integration: Easily connects to external notification or reporting systems

---

3. CampaignExecutionConsumer

Purpose: Periodically queries the DB, finds CampaignExecution entries with status NEW, and initiates execution.

Technical Details:

• Uses @Scheduled + Shedlock to ensure single-instance task launching

• Uses getSync(id) for locked access to prevent race conditions

• Uses ttb.requiresNew() to run execution in a new transaction, independent of outer context

• Updates execution status, applies filtering, and handles errors

---

🔐 Advantages of This Approach

• Decouples task creation from execution

• Ensures safe execution with transaction isolation

• Tolerant to crashes and service restarts

• Scales well in clustered environments

---

⚠️ Potential Risks and Mitigations

• Race conditions → Mitigated via getSync(...) and Shedlock

• Duplicate processing → Avoided through status flags and atomic updates

• Task pileup if consumers are slow → Addressed by monitoring and horizontal scaling

---

📘 Where Else This Pattern Can Be Used

The Producer–Database–Consumer model is widely used across timveroOS, including:

• Offer generation

• Bulk application processing

• Data import/export

• Periodic reconciliations and calculations