From the Architect’s Journal — Apache Hardening, Uploads, and Bugs

There has never been a case where introducing a completely new feature or deploying a new configuration worked flawlessly on the first try.
This time was no different.

Context

After attending a training by Maciej Szymczak at Sekurak Academy on web server hardening, I decided to get some hands-on practice in my own environment. Since I maintain a small web system, choosing a target was straightforward — Apache 2.4 acting as a reverse proxy.

Apache, mpm_event, cache… and mod_security?

I started with the basics: tuning the server for better resource utilization and load distribution.
While Apache does not provide direct control over CPU affinity, it gives fine-grained control over how processes and threads are managed — how many are spawned, how long they live, and how they behave under load. This is handled through the mpm_event module, which replaced mpm_prefork in my setup.

Next came caching.

This part required more caution. My system is not a static content platform, and it enforces strict access control policies. Proxy-level caching is not inherently session-aware, which means it can unintentionally expose protected resources if not configured carefully.

For example, caching a URL like /preview/<id1>/<id2> without proper safeguards could allow someone to discover cached content through brute force attempts. Unless caching is explicitly bypassed for certain HTTP methods or headers, sensitive data could leak.

As a result, I introduced caching only where it was safe to do so. Even partial caching still provides noticeable performance benefits.

Then there was mod_security.

It was not part of the training, but it is part of my stack — and it became the starting point for debugging when, after hardening the server, I suddenly found myself unable to upload larger files.

Larger files… or not?

In parallel, the application had already been updated to support larger uploads. Request size limits, read timeouts, and proxy timeouts were all adjusted accordingly.

Yet uploads were still failing — typically after the request had already been buffered.

The first suspect was mod_security, which logged warnings about rule 20004 — unmatched multipart boundary. That initially seemed plausible. However, even after disabling the rule (or switching it to pass/log mode), the issue persisted.

The real cause turned out to be far more familiar: the application layer.

Earlier, I had implemented a standard JWT-based authentication flow with refresh tokens. The frontend was configured to refresh tokens whenever a request returned a 401, and it also periodically checked system status in the background.

The problem was the short lifespan of the access token — 2 minutes.

For uploads larger than ~10MB, especially over LTE or unstable WiFi, the transfer time could easily exceed that limit — sometimes by a large margin. What happened in practice was:

the request was fully buffered on the server side,
the access token expired during that time,
the request reached the backend with an invalid token,
the server returned 401,
and the entire upload failed.

In short: security worked exactly as designed — just not in this scenario.

The fix

The solution was to introduce a dedicated mechanism for uploads:

a separate, long-lived upload token (TTL ~1 hour),
stored in cache,
issued only after an explicit, authenticated request,
with support for refresh on the frontend if needed.

At the same time:

uploads remain restricted to authenticated users,
session timeouts are still enforced to prevent abuse (e.g., DoS),
and the standard authentication model remains unchanged.

Takeaways

This issue was ultimately caused by a lack of proper testing — but also by stacking multiple significant changes without validating the system as a whole.

Two key lessons stand out:

Do not combine major infrastructure changes with substantial application-level refactoring without proper validation.
Design security mechanisms with real-world usage patterns in mind, including long-running operations like file uploads.