Preventing OOM: Optimizing Laravel Horizon and Deployments for Low-RAM Servers

Introduction

In the Breniapp project, we recently tackled a critical infrastructure challenge: out-of-memory (OOM) crashes during deployments on a server with limited RAM. This is a common pain point for many applications running on constrained environments, where simultaneous resource-intensive tasks can quickly exhaust system memory. Our experience highlights the importance of right-sizing your queue workers and hardening your deployment processes.

The Problem

The root causes of our OOM crashes were multi-faceted. Firstly, we discovered a duplicated Laravel Horizon daemon running on Forge, effectively doubling our worker footprint. Beyond that, our maxProcesses configuration for Horizon was overly aggressive, leading to potentially 40 workers running concurrently, each consuming significant memory. Compounding these issues was an inefficient deployment script. It indiscriminately copied vendor/ and node_modules/ directories even when lockfiles hadn't changed, leading to unnecessary I/O. Crucially, it failed to pause Horizon workers before triggering demanding build steps like composer install and npm run build, which are memory-intensive operations.

The Solution: Right-Sizing Horizon and Hardening Deployment

To mitigate these issues, we implemented a series of targeted optimizations: First, we eliminated the redundant Horizon daemon. Then, we significantly reduced the maxProcesses configuration for production, scaling it down to a more realistic 2/2/2, alongside allocating 128MB of memory per worker. This immediately curbed the runaway memory consumption.

A key optimization was consolidating our queue structure. We had five separate social-* queues, which implicitly encouraged more worker processes. By merging these into a single social queue, we simplified management and reduced the overall worker footprint. All relevant jobs, such as FetchSocialDataJob and PublishContentJob, were updated to dispatch to this new consolidated queue.

<?php

namespace App\Jobs;

use Illuminate\Bus\Queueable;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Queue\SerializesModels;

class FetchSocialDataJob implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;

    public function __construct(public int $userId)
    {
        $this->onQueue('social'); // Dispatching to the consolidated 'social' queue
    }

    public function handle(): void
    {
        // Logic to fetch social data
        logger()->info("Fetching social data for user: {$this->userId}");
    }
}

This example demonstrates how a job is assigned to the new, consolidated social queue using the onQueue() method within its constructor.

Finally, we hardened our deployment script (deploy/forge-deploy.sh). We introduced horizon:pause at the beginning of the pre-activate stage, ensuring workers halt gracefully during resource-intensive build steps, with a trap to resume them even if the deploy fails. We also added logic to conditionally reuse vendor/ and node_modules/ only if composer.lock and package-lock.json haven't changed, drastically cutting down on unnecessary I/O and build times. npm ci was configured with --no-audit --no-fund for faster, cleaner installs, and NODE_OPTIONS=--max-old-space-size=384 was set to prevent Node.js from consuming excessive memory during compilation.

Immediate Impact

These changes immediately stabilized our deployments on the low-RAM server. The OOM crashes ceased, and the overall deployment duration was reduced due to more efficient build processes. Consolidating queues also simplified queue monitoring and management, ensuring a smoother operation of background tasks.

Lessons Learned for Efficient Deployments

When working with resource-constrained environments or high-load applications, consider these practices:

1. Monitor Your Daemons: Regularly check for duplicate or rogue processes that consume resources unnecessarily.
2. Right-Size Your Workers: Tailor your queue worker maxProcesses and memory allocations to your server's capacity and workload. Don't overprovision.
3. Consolidate Queues: Where logical, consolidate multiple small queues into fewer, broader categories to simplify management and potentially reduce worker count.
4. Harden Deployment Scripts: Integrate graceful pauses for critical services (like Horizon) during builds, and implement intelligent caching for dependencies to avoid redundant operations.
5. Optimize Build Steps: Configure build tools (Composer, NPM, Vite) for minimal memory usage and faster execution during deployment.

Key Insight

Balancing application functionality with infrastructure constraints requires proactive optimization, especially in areas like background job processing and deployment pipelines. Small efficiencies in configuration and scripting can yield significant stability gains on resource-limited systems.