How we solved database lock contention, local SQLite sync, and event-driven Kafka reconnection for 450+ retail store checkouts.
When building point-of-sale software for high-volume retail chains, network latency and internet outages are non-negotiable failure points. A 5-minute internet drop at a busy hypermarket can paralyze checkout queues, resulting in lost revenue and customer frustration.
## The Architectural Challenge
Traditional cloud POS solutions rely on constant HTTP connections to a centralized server. If the connection drops or latency spikes above 500ms, transaction UI locks up.
To overcome this, we designed **Retail OS** around an **Offline-First Event Sourcing Architecture**.
### Key System Components
1. **Local Embedded Ledger**: Every checkout terminal runs an isolated SQLite database wrapped in WAL (Write-Ahead Logging) mode.
2. **Local Write Loop**: POS transactions are committed locally in <15ms without waiting for network confirmation.
3. **Kafka Event Bus Synchronization**: Once connected, an out-of-process daemon streams local transactions as immutable events to an AWS EKS Kafka cluster.
```typescript
// Event Schema for POS Local Sync
interface POSTransactionEvent {
transactionId: string;
storeId: string;
terminalId: string;
timestamp: number;
items: Array<{ sku: string; quantity: number; unitPrice: number }>;
paymentMethod: 'CASH' | 'CARD' | 'UPI';
checksum: string;
}
```
## Resolving Conflict Resolution (CRDTs)
When 450 stores reconnect simultaneously after a regional network blackout, handling inventory reconciliation without race conditions requires strictly deterministic rules. We utilized **Conflict-Free Replicated Data Types (CRDTs)** for inventory count aggregations.
### Key Takeaways
- Never block local UI execution on remote network calls.
- Immutable event streams eliminate data overwrite corruption.
- Blue/green database read replica routing keeps analytical queries off master transaction nodes.