Load Balancing in the Cloud: Layer 4, Layer 7, API Gateways, and Beyond

Conceptually, load balancing is easy to understand: network traffic is distributed across multiple servers to ensure no single server becomes a bottleneck — if one server is busy or down, the load balancer directs traffic to others. However, when it comes to practices in cloud environments, you might find the services offered by cloud providers confusing. For example, in AWS, do you know the difference between Network Load Balancer (NLB), Application Load Balancer (ALB), and API Gateway? What about Route 53 or other third-party services, such as Akamai or Apigee? In this post, I will categorize these products by the OSI layer at which they operate, specifically Layer 4 (Transport) or Layer 7 (Application). I will outline their similarities and differences, and discuss when to use each.

OSI Layers and Load Balancing

OSI Layers and Load Balancing: Load balancers can operate at different layers of the OSI (Open Systems Interconnection) model, notably at Layer 4 (Transport) or Layer 7 (Application). A Layer 4 (L4) load balancer routes traffic based on network information – IP addresses and TCP/UDP ports – without inspecting the actual content of the packets. In contrast, a Layer 7 (L7) load balancer operates at the application level, making decisions based on content such as HTTP headers, URLs, and cookies. This fundamental difference underpins the capabilities and optimal use cases of each type.

L4 Load Balancing – Network-Level, Faster but Simpler

It forwards packets at the transport layer and simply uses information like source/destination IP and port, and a load-balancing algorithm (e.g., round-robin, hash-based, least connections) to distribute requests. Because it doesn’t “look” inside the traffic (i.e., decrypt or parse messages), L4 balancing adds minimal latency and can handle very high throughput (e.g., millions of requests per second). However, it cannot make content-based routing decisions. For example, it can’t send /api/* requests to one server group and /images/* to another. AWS Network Load Balancer (NLB) and Azure Load Balancer are such L4 load balancers.

Use Cases for L4 Load Balancing

L4 load balancers are best suited for scenarios where raw speed and network-level distribution are needed without application-layer smarts. They also shine for non-HTTP protocols. Typical use cases include real-time streaming and IoT backends.

Real-Time Streaming: Fast-paced online games, live media streaming, or VoIP often use UDP or custom TCP protocols for communication. These applications demand ultra-low latency and high throughput, often accepting imperfect reliability. For example, the audiences watching the World Cup final usually care more about watching the game in real time than receiving every single video frame perfectly. A few dropped packets might result in minor glitches, but most users won’t mind as long as they stay close to the live action. L4 load balancers are ideal for such applications.

IoT and Telemetry Backends: The Internet of Things often involves massive numbers of devices sending small, frequent messages or maintaining long-lived connections to send sensor data. IoT protocols such as MQTT or CoAP are typically not HTTP-based, and the services need to handle potentially millions of simultaneous connections. For instance, in a smart city, many smart meters report electricity usage to the utility company. The backend service must manage millions of simultaneous connections, receiving real-time data from these devices. L4 load balancers are a better fit for this scenario, as they can manage high volumes of connections and distribute them across backend services without being hindered by application-layer logic.

L7 Load Balancing – Application-Level, Slightly Heavier but Content-Aware

It acts as a proxy that can inspect the contents of each request (e.g., the URL path, host name, headers, or message body) to make more granular decisions. It can terminate client connections (for HTTP/HTTPS), read or modify the request, and then forward it to the appropriate backend based on rules. This allows features like URL-based routing, header-based routing, SSL/TLS termination, cookie-based session affinity, and even injecting custom headers or cookies for advanced scenarios. The trade-off is the slight latency introduced when inspecting the processing requests. Besides, L7 load balancers typically handle only HTTP/HTTPS traffic, not arbitrary TCP/UDP traffic. AWS Application Load Balancer (ALB) and Azure Application Gateway are such L7 balancers.

Use Cases for L7 Load Balancing

Advanced Routing for Web Apps & Microservices: Microservices-based web application is the poster child use case for L7 balancers, as it often requires path-based, host-based, or header-based routing. For example:

  • /api/* goes to the API servers; /web/* goes to the front-end servers
  • /orders/* and /users/* routes to the Order and Users service, respectively
  • app1.yourdomain.com and app2.yourdomain.com forwarded to different backends
  • Images or videos go to the optimized storage
  • Requests with X-Canary HTTP header go to a beta deployment slot

SSL Termination and Web Security Enforcement: Many organizations use L7 load balancers as a central point for SSL termination (offloading) and security checks for web traffic. In this use case, the load balancer handles all incoming HTTPS connections, decrypting them using certificates you’ve installed on the LB. The benefits include: (a) You can manage all your SSL/TLS certificates in one place, simplifying renewal and compliance. (b) Your backend servers can receive traffic in unencrypted form (HTTP), offloading the CPU work of encryption/decryption. Building on this, once the traffic is terminated at the L7 load balancer, you can enforce various web security measures. For example, you can enable a Web Application Firewall (WAF) at this layer to inspect incoming requests for SQL injection attempts, XSS attacks, or other malicious patterns.

In summary, whenever you need to make traffic routing decisions using anything above the IP and port (like URI paths, hostnames, languages, device type in headers, etc.), or you need to combine load balancing with SSL management and request-level security or manipulations (like adding standard HTTP response headers or rewrites), a L7 load balancer is the right tool. It provides the central control point for these concerns across all your web servers.

What About API Gateways?

Products like AWS API Gateway, Azure API Management, or Apigee do route and manage traffic to backend services, but they are not load balancers because they do not distribute or balance the loads. Such API management services operate at Layer 7 and offer many advanced features not available or only partially supported in L7 load balancers, such as:

  • Authentication & Authorization (OAuth2, JWT validation, API keys)
  • Rate limiting/throttling
  • API key/usage plans or quotas
  • Request/response transformation
  • API versioning and lifecycle management
  • Built-in caching
  • GraphQL support

In short, you use:

  • API Gateway for full API lifecycle management
  • Layer 7 load balancer for intelligent HTTP routing
  • Layer 4 load balancer for high-speed, protocol-agnostic load balancing

Below are typical request flows in which you place each component.

Example request flow and traffic management

The table below summarizes the core distinctions and best-fit scenarios for layer 4, layer 7 load balancer, and API gateway:

FeatureL4 Load BalancerL7 Load BalancerAPI Gateway
OSI LayerLayer 4 – TransportLayer 7 – ApplicationLayer 7 – Application
Primary ProtocolsTCP, UDPHTTP, HTTPS, DNS, FTP, SMTPHTTP, HTTPS, WebSocket, gRPC
Traffic AwarenessIP and PortContent-aware (path, host, headers)Full API-aware (method, version, header, payload)
Routing LogicIP/Port-basedPath- and host-based routingMethod-based, version-based, policy-based
Use Case FocusHigh-throughput, low-latency, simple traffic distributionSmart routing, content switching, SSL terminationAPI management, security, microservices, monitoring
SSL/TLS TerminationOptional (basic)Yes (with offload)Yes (with additional features like JWT validation)
AuthenticationNoNo or basicYes and buit-in (OAuth2, JWT, API key)
Security FeaturesDDoS layer, NSG/firewall integrationWAF integrationAuthentication, threat protection, rate limiting, payload validation
ExamplesAWS NLB
Azure Load Balancer
AWS ALB
Azure Application Gateway
AWS API Gateway
Azure API Management
Best ForGaming, IoT, raw TCP/UDP or non HTTP/S workloadsHTTP/S applications needing smart routing (web apps, microservices)Microservices, API-first architectures, security, analytics

Other Traffic Management Services: DNS Load Balancing, Akamai

DNS load balancers such as AWS Route 53 and Azure Traffic Manager perform global traffic distribution at the DNS level. They make DNS-based routing decisions, returning different IP addresses to users based on defined rules (geo-based, latency-based, or weighted routing). They also perform health checks to route traffic only to healthy endpoints. However, they do not proxy or inspect the traffic.

On the other hand, you may have heard of Akamai and wonder how it is related to the concepts we discussed above. Essentially, Akamai is a CDN (Content Delivery Network) and edge platform that combines DNS load balancing, API gateway, L7 load balancing, WAF, and other features, depending on how it is configured. The term “edge servers” (also known as Points of Presence, or PoP) refers to distributed servers located globally and close to users. These edge servers serve as the first point of contact that is physically closest to your end users, and make traffic steering decisions — terminating, inspecting, routing, caching, transforming HTTP/S requests, you name it — before a request crosses the long‑haul network to your VPC or on‑prem DC. You can choose to configure such L7 reverse proxies at the edge using purely AWS or Azure without Akamai, but that typically requires four or more separate services.

Jim
Jim

Site reliability engineer working in a global gaming and entertainment company

Articles: 18

Leave a Reply

Your email address will not be published. Required fields are marked *