Optimizing Horizon Worker Limits to Prevent Out-of-Memory Errors
Introduction
The Reimpact/platform project experienced production crashes due to out-of-memory (OOM) errors. These errors stemmed from the Horizon queue worker consuming excessive memory, particularly on smaller instance sizes.
The Challenge
The default configuration for Supervisor-2, responsible for managing Horizon workers, was allocating substantial memory per worker process.