There is a lot written about the infrastructure side of load balancing - the Azure setup, Redis, SignalR and so on. But there is a question all of that setup doesn't really answer. Is your actual code ready to run on more than one instance? That is what I want to look at here, and it matters just as much whether you are building a site or shipping a package that ends up on someone else's load balanced site.
First a quick bit of background. Load balancing means running multiple instances of the same application behind a load balancer that distributes incoming requests across them. The motivation is simple, you can scale out by adding instances when traffic spikes, and you get resilience because if one instance dies or restarts mid-deployment, the others keep serving requests without the user noticing.
Once you have more than one instance, though, there are two ways the load balancer can decide which instance handles each request.
Sticky sessions
When a request first arrives, the load balancer picks an instance and returns an affinity cookie with the response. Every subsequent request from that user carries the cookie, so the load balancer pins them to the same instance. The upside is that each user's requests land in the same place, which makes things like in-memory session state work without any special treatment. The downside is uneven distribution - one instance can end up with a lot of "sticky" users while others sit mostly idle. And sticky isn't forever, if your auto-scaling rules add or remove instances, some users get reassigned to a different instance whether they like it or not.
Stateless
No cookies, no pinning. Each request is routed independently - there are a few strategies for deciding where it goes, like round-robin or picking the least busy instance, but the exact mechanism doesn't matter much here. What matters is that any request can land on any instance, regardless of where that user's previous request went. Distribution tends to be more even, which is the main appeal. The trade-off is that you can never assume two consecutive requests from the same user land on the same server.
Either way, once a load balancer is in front of your app, your code can no longer assume there is only one server running it.
A note for package authors
Some of the rules in this post are only strictly required when running without sticky sessions. But code that works under stateless distribution also works under sticky sessions - the reverse is not true. As a package author you have no control over how the sites running your package are hosted, so write for stateless - it is the lowest common denominator that works everywhere. {% endcard %}
Keep state in a shared datasource
Imagine a webshop that keeps each visitor's basket in memory, in a static dictionary keyed by a basket ID. Add an item and it works. Add the next item and that request happens to land on another instance, which has its own empty dictionary - so the basket looks empty and the first item is gone. The user re-adds it, the following request bounces back to the instance that did have the original, and now they are staring at a duplicate. Nothing throws and nothing is logged, the basket just behaves differently on every click depending on which instance answered.
Anything that must survive beyond a single request needs to live in a store that every instance can reach - the database, blob storage, or a distributed cache. Statics, singletons holding data, and sessions in their default in-memory form are all off limits for anything you can't afford to lose or that needs to be consistent across instances. Sessions aren't banned outright, though - register a distributed cache and they're automatically stored in it instead, which we'll get to later in the post.
// Breaks on load balancing: this dictionary only exists on one instance
// (a field inside some service class)
private static readonly Dictionary<Guid, Basket> _baskets = new();
// Works everywhere: backed by a store all instances share
public class BasketStore(IDistributedCache cache)
{
public async Task SaveAsync(Guid basketId, Basket basket) =>
await cache.SetStringAsync(
$"basket:{basketId}",
JsonSerializer.Serialize(basket),
new DistributedCacheEntryOptions { SlidingExpiration = TimeSpan.FromMinutes(30) });
public async Task<Basket?> GetAsync(Guid basketId)
{
var json = await cache.GetStringAsync($"basket:{basketId}");
return json is null ? null : JsonSerializer.Deserialize<Basket>(json);
}
}
To be clear though, in-memory caching of immutable or re-derivable data is still fine - that's exactly what caches are for. If you fetch a product catalogue from the database and cache it in memory for five minutes, the worst case is a stale read or a redundant database call when a new instance starts up. That's acceptable. The test is whether stale or missing data causes a real problem - if it does, it belongs in a shared store.
It is worth being clear about how sticky sessions fit in here, because it is tempting to think they make this go away. They do not. With sticky sessions a visitor would not jump instance on every request the way the basket example does - the affinity cookie keeps them pinned to one instance. But that pinning only lasts as long as the instance does. When it gets removed in a scale-down, or recycled during a deploy, the cookie points at an instance that is no longer there and the load balancer hands the visitor to a fresh one that has never seen their basket. The exact same problem happens, just far less often. Sticky sessions make it rarer, not impossible, which is why the basket belongs in a shared store either way.
Make integrations idempotent
Imagine the checkout on that same webshop. When the customer hits Pay, a handler charges their card and then marks the order as paid. On a single instance this fires exactly once and all is well. On a load balanced site the picture changes. The charge succeeds on instance A, but the response times out before it gets back to the browser. The customer clicks Pay again, the retry lands on instance B with no memory of the first attempt, and their card is charged a second time for the same order. An automatic retry somewhere in the stack, or a scaling event moving the request mid-flow, does exactly the same thing - anything that runs the charge twice charges the customer twice. Nothing throws, the money just leaves their account twice and the support tickets start arriving.
Design every integration so that running it twice has the same effect as running it once. For something like a payment, the cleanest way there is an idempotency key - a value you generate once and send along with the charge, so the provider recognises a retry and collapses it into a single charge rather than making a new one. The order number is a natural fit, it already exists and is unique to the order.
public async Task ChargeAsync(Order order)
{
// Same key on a retry means the provider charges once, not twice
await _paymentProvider.ChargeAsync(
amount: order.Total,
idempotencyKey: order.OrderNumber);
}
Not every system hands you an idempotency key. When it doesn't, the equivalent is to look before you act - check whether this order already has a payment and skip it if so, or use a natural key and upsert rather than blindly insert. Same idea, just enforced on your side instead of theirs.
Idempotency is the safety net the rest of this post keeps falling back on. When requests can land anywhere and retries are normal, "safe to run twice" is what keeps integrations correct.
Don't work off files on disk
A package ships with an import feature. Triggering an import writes the pending work to App_Data/import-queue.json, and a background job on a timer wakes up, reads the file, and processes the rows. On a single-instance dev environment this works every time. Add a load balancer and what happens next depends on your hosting, and neither outcome is good.
If the instances don't share a file system - containers being the obvious case - the write lands on instance A while the background job happens to run on instance B, which has its own disk and has never seen the file. The job finds nothing, exits cleanly, and the import never runs.
If the instances do share a file system - the default on Azure App Service, where /home is mounted across every instance - you get the opposite problem. Every instance's timer fires, every instance sees the same file, and they all start processing it at once. The same rows get imported several times over, and now you are dealing with duplicate records and whatever database or distributed-cache locking you bolt on afterwards to stop the instances trampling each other.
The file on disk was never the right tool for this. Data that more than one instance needs to read belongs in a shared store - the database, blob storage, or the distributed cache - and work that needs picking up belongs on a real queue, where each message is handed to exactly one consumer, rather than a file that every instance polls.
// Breaks: a local file that a per-instance background job polls
// (App_Data is illustrative - any local path has this problem)
await File.WriteAllTextAsync(
Path.Combine(_env.ContentRootPath, "App_Data", "import-queue.json"), json);
// Works: hand the work to a queue, so each message is
// processed by exactly one instance
await _queueClient.SendMessageAsync(json);
The same logic applies to media. On a load balanced Umbraco site, media should live in a remote file system provider - the Azure Blob Storage provider is the standard choice.
Use IDistributedCache - but mind what hits it
I've mentioned the distributed cache a couple of times now as the right place to put shared state. Let's get concrete, because registering a real IDistributedCache does more than just give you a place to store things.
If your site is load balanced, you really should swap out ASP.NET Core's default in-memory cache implementation for something shared - Redis is the usual choice, registered as your IDistributedCache with AddStackExchangeRedisCache. That one registration solves several things at once. ASP.NET Core sessions automatically back onto it, and Umbraco's content and media cache is built on HybridCache which picks up any registered IDistributedCache as its second-level cache.
With the cache registered, the simplest way to actually use it is HybridCache (AddHybridCache()), which layers an in-memory L1 over the Redis L2 you just set up. A single GetOrCreateAsync checks L1, falls back to L2, and only runs your factory on a full miss - with built-in stampede protection, so a miss under load doesn't fire the same query a hundred times at once:
public class ProductService(HybridCache cache, IProductRepository repository)
{
public async Task<Product?> GetAsync(int id, CancellationToken ct = default) =>
await cache.GetOrCreateAsync(
$"product:{id}",
async token => await repository.GetByIdAsync(id, token),
new HybridCacheEntryOptions
{
Expiration = TimeSpan.FromMinutes(30), // L2 in Redis, shared by every instance
LocalCacheExpiration = TimeSpan.FromMinutes(1) // L1 in memory, kept short to limit desync
},
tags: ["products", $"product:{id}"],
ct);
// Call this when a product changes - clears this instance's L1 and the shared L2
public ValueTask InvalidateAsync(int id) =>
Mind what hits it
The catch is that every read against IDistributedCache is a network round-trip, and things can hit it far more often than you'd expect. Sessions are the classic example. The session loads lazily - the round-trip only fires when something accesses HttpContext.Session - but in practice that "something" is often a shared layout component or middleware that runs on every page, which means every page view loads session state over the wire (and writes it back whenever it changed). Multiply that by every concurrent user and Redis becomes a hot dependency on your hottest path. A slow or unavailable Redis instance stops looking like a caching problem and starts looking like a site outage.
A few things to keep in mind:
- Only enable session middleware if you actually use sessions. If you added it speculatively years ago and nothing reads it, remove it. And if you do use sessions, keep the payloads small - the serialized session state travels over the wire on every hit.
- Don't reach for the distributed cache for data that's cheap to re-derive per instance. In-memory caching and HybridCache's L1 layer are there for a reason. The product catalogue from earlier, a config value, a list of countries - all fine to keep in memory per instance with a short TTL.
- Be careful once that in-memory cache holds mutable data. Invalidating it on instance A doesn't touch instance B's copy, so they desync until B's entry expires. HybridCache doesn't save you here - invalidating clears the calling instance's L1 and the shared L2, but other instances keep their stale L1 copy until it expires locally. Keep the local expiration short when a little staleness is harmless, and when it isn't, broadcast the invalidation to every instance - Redis pub/sub, or in Umbraco a custom ICacheRefresher dispatched through the DistributedCache service.
- Watch for per-request reads of values that change rarely. Feature flags and application settings are common offenders. Code that calls cache.GetStringAsync("feature-flags") on every request will hammer Redis even though those values might change once a week. Cache them in memory with a short TTL instead, and let the distributed cache be the source of truth rather than the hot path.
- Call the cache asynchronously wherever you can. Synchronous calls into Redis park a thread while they wait on the network, and a burst of traffic can drain the thread pool faster than it grows back - the classic thread pool starvation that shows up as Redis timeouts even when Redis itself is healthy. Prefer the async APIs (including the async session methods), and if bursts still bite, raise the minimum thread count so the pool can absorb them.
That last point matters most with sessions, where it is easy to block by accident. Load the session asynchronously up front instead of blocking a thread on the first synchronous access:
// LoadAsync pulls the session from Redis into a local snapshot - async, no blocking
await httpContext.Session.LoadAsync();
// Get and Set work against that local snapshot only - nothing hits Redis here
var count = httpContext.Session.GetInt32("basketCount") ?? 0;
httpContext.Session.SetInt32("basketCount", count + 1);
// CommitAsync writes the changes back to Redis asynchronously
// (the session middleware also commits automatically at the end of the request)
await httpContext.Session.CommitAsync();
The HttpClient handler recycling gotcha
Here's a failure mode that shows up whenever your code calls a service sitting behind a sticky-session load balancer - think an external API, a headless commerce backend, or for that matter a load balanced Umbraco site. Your side doesn't even need to be load balanced - a plain single-instance app making these calls hits it just the same.
You reach for IHttpClientFactory, attach a CookieContainer to the primary handler (via ConfigurePrimaryHttpMessageHandler) so the affinity cookie flows on every call, and things look fine in testing. Say you are uploading a large file to the API in chunks - you open an upload session, send the chunks one after another, then tell the API to finalise it. Every request lands on the same remote instance, and it works because that instance is the one holding your half-uploaded file between calls.
The trap is that IHttpClientFactory recycles its message handlers. The default handler lifetime is two minutes. The CookieContainer lives on the handler - not on the named client or your service - so when the handler is recycled after two minutes, the container goes with it. Your next outgoing request reaches the remote load balancer without an affinity cookie, gets treated as a brand-new client, and can land on a completely different remote instance - one that has never seen your upload session. Nothing throws; the remote service just has no idea what you're talking about. This isn't an obscure edge case either - Microsoft's own docs warn about it and go as far as recommending you avoid IHttpClientFactory altogether if your app depends on cookies.
You can push the window out by configuring a longer lifetime:
builder.Services.AddHttpClient("uploads", client =>
{
client.BaseAddress = new Uri("https://api.example.com");
})
// Default is 2 minutes - any cookies accumulated in that window
// are lost when the handler is recycled
.SetHandlerLifetime(TimeSpan.FromMinutes(10));
But ten minutes is still a window, not a guarantee - you've changed the frequency of the failure, not the nature of it. And if you're thinking of setting the lifetime to something enormous to make the cookie effectively permanent - don't. Handler recycling is how the HTTP stack picks up DNS changes when the remote service scales or fails over. Disabling it trades one class of failure for another.
The real fix is the same one the idempotency section already handed you. Don't design a multi-request flow that depends on remote state surviving between calls. If the remote instance can disappear mid-operation - and from your side it can, at any moment - then the operation needs to be resumable or idempotent. Store the operation ID somewhere durable on your side, pass it as a correlation key on every call, and make sure the remote service can pick up from wherever it left off when your request lands on a fresh instance. "Safe to run twice" covers instance jumps in both directions.
And if you build a long-lived HttpClient yourself with SocketsHttpHandler and PooledConnectionLifetime, the same recycling behaviour applies there too, and the fix is the same - stop depending on affinity for correctness.
Wrapping up
Before you flip the switch on load balancing, run through these five:
- State that must survive a request lives in a shared datasource - not a static field, not a singleton holding data, not in-memory session.
- Every integration is idempotent - safe to run twice without creating duplicates or corrupt state. An idempotency key, a natural-key lookup, or an upsert gets you there.
- Nothing uses the local filesystem to share data or coordinate work between instances - depending on hosting it's either not shared at all or shared and contended. Data belongs in a shared store, work belongs on a queue.
- A real IDistributedCache is registered - called asynchronously, and not hammered with high-frequency reads of data that barely changes. Sessions are the first thing to audit.
- No code depends on session affinity for correctness - not a remote service you're calling, not a multi-step operation that relies on landing on the same remote instance each time. Handler recycling will eventually break it.
These patterns cost almost nothing on a single-instance site, and they're much easier to build in from the start than to unpick later when the site suddenly needs to scale. For package authors especially, writing for stateless distribution means your package simply works - regardless of how the site running it is hosted.
Thanks for following along, I hope it was helpful! Please let me know if this was useful, and feel free to reach out to me at jmh@umbraco.dk with any questions or feedback 🙂