GEP-1619: Session Persistence and Session Affinity¶
- Issue: #1619
- Status: Provisional
(See definitions in [GEP Status][/contributing/gep#status].)
TLDR¶
This GEP proposes a definition of session persistence and session affinity as the foundation of future session persistence related improvements. This GEP also outlines the ways session persistence could be achieved by implementations.
Goals¶
- Define session persistence and session affinity to establish a common language
- Identify differences in session persistence functionality between implementations
Non-Goals¶
- Mandate a default or expected session persistence functionality for implementations
Introduction¶
Defining Session Persistence¶
Session persistence is when a client request is directed to the same backend server for the duration of a "session". It is achieved when a client directly provides information, such as a header, that a proxy uses as a reference to direct traffic to a specific server. Persistence is an exception to load balancing: a persistent client request bypasses the proxy's load balancing algorithm, going directly to a backend server it has previously established a session with.
Session persistence enables more efficient application workflows: 1. Better performance: Maintaining a single session allows a server to cache information about a client locally reducing the need for servers to exchange session data and overall storage needs. 2. Seamless client experience: Clients can reconnect to the same server without re-authenticating or re-entering their information.
Some of the concerns of session persistence are the duration and expiration of the session, security of the transaction stream, and storage of the context or state.
Session affinity, not to be confused with session persistence, uses an existing attribute of the request to consistently send to the same backend. Session affinity can be considered a weaker form of session persistence: it is not guaranteed to persist a connection to the same backend server if certain attributes of the request or the backends are changed.
Security and Privacy Implications¶
Session persistence can introduce security and privacy vulnerabilities if not properly implemented. These vulnerabilities can include:
- Session hijacking: Attackers intercepting or predicting a valid session token to gain unauthorized access.
- Session fixation: Attackers setting a client's session ID to a known value, which they can then use to hijack the session.
- Session replay attacks: Attackers capturing and resending a client's message with a valid session ID.
- Data leakage: Attackers can exploit sensitive session information cached on servers if not properly secured.
- Denial of service attacks: Attackers can use up server resources by creating and maintaining large numbers of sessions.
To mitigate these security concerns, it is important to implement session persistence using secure practices, such as using strong session ID generation algorithms, implementing session timeouts, encrypting sensitive data, and monitoring server resources for unusual activity.
IP address reuse may also be a security or privacy concern when using session persistence or session affinity. If Kubernetes reuses an IP address of previously shutdown pod, the new pod may receive session persistent traffic meant for the old pod.
Session affinity introduces fewer security and privacy vulnerabilities since there are no session tokens to protect or exploit.
Achieving Session Persistence¶
Session persistence is achieved using attributes residing in the application layer. The following are mechanisms for achieving session persistence:
1. Cookie-Based Session Persistence
The most common mechanism is by using cookies (described by RFC6265) with the set-cookie HTTP response header. A client will use the provided value in the set-cookie response header in a cookie request header in subsequent requests. Proxies can use this cookie header to maintain a persistent connection to a single backend server on behalf of the client.
2. Header-Based Session Persistence
Header-based stateful sessions are achieved by a backend or gateway providing an HTTP response header and the client using the same header in subsequent HTTP requests. Proxies can use this header to maintain a persistent connection to a single backend server on behalf of the client.
3. URL-Encoded Session Persistence
Session information can be also encoded into the request URL to establish a persistent session. The server rewrites the client's URL to encode the new session information automatically. The server then decodes the session information from the URL to identify the session.
Session Initiation
For both header-based and cookie-based sessions, either the gateway or the backend server can initiate establishing the session via the appropriate header or set-cookie attributes. The following rules apply based on who initiates the session:
- If the backend initiates the session, the gateway should allow this and not force persistence connections, unless specifically configured to. The gateway may decode and alter the cookie established by the backend to achieve session persistence.
- If the gateway initiates the session, the backend will be presented with session attributes regardless if it enabled them.
The client application also plays in a role in session initiation. If a client doesn't want a request to belong an existing session, it can remove the session cookie or identifier to signal the gateway or backend to recreate the session.
Achieving Session Affinity¶
While session persistence uses attributes in the application layer, session affinity often uses, but is not limited to, attributes below the application layer. Session affinity doesn't require a session identifier like session persistence (e.g. a cookie), but instead uses existing connection attributes to establish a consistent hashing load balancing algorithm.
Session affinity can be achieved by deterministic load balancing algorithms or a proxy feature that tracks IP-to-backend associations such as HAProxy's stick tables or Cilium's session affinity.
The Relationship of Session Persistence and Session Affinity¶
Although session persistence and session affinity may appear to be quite similar, it is important to recognize and understand their distinctions. As mentioned above, their distinctions are defined by what information they use to establish session consistency and whether consistency is guaranteed.
We can also examine how session persistence and session affinity functionally work together, by framing the relationship into a two tiered logical decision made by the data plane: 1. If the request contains a session persistence identity (e.g. a cookie or header), then route it directly to the backend it has previously established a session with. 2. If no session persistence identity is present, load balance as per load balancing configuration, taking into account the session affinity configuration (e.g. by utilizing a hashing algorithm that is deterministic).
This tiered decision-based logic is consistent with the idea that session persistence is an exception to load balancing. Though there are different ways to frame this relationship, this design will influence the separation between persistence and affinity API design.
Implementations¶
In this section, we will describe how implementations achieve session persistence and affinity, along with a breakdown of related configuration options. Input from implementations is appreciated to complete this information.
In the following tables, we will example two types of APIs: 1. Dataplane APIs 2. Implementation APIs
Generally, the implementation API programs the dataplane API; however these two are not always clearly separated. The two types of APIs can use different API structures for configuring the same feature. Examining the dataplane APIs helps to remove the layer of API abstraction that implementations provide. Removing this layer avoids situations where implementations don’t fully implement all capabilities of a dataplane API or obfuscate certain configuration around session persistence. On the other hand, examining implementation APIs provides valuable data points in what implementations are interested in configuring.
Session Persistence
| Technology | Technology Type | Session Persistence Type | Configuration Options | Configuration Association (Global, Gateway, Route, or Backends) | Notes |
|---|---|---|---|---|---|
| Acnodal EPIC | Implementation (Envoy) | N/A | Supports Gateway API Only* | N/A | *Acnodal Epic solely uses Gateway API; therefore, it doesn’t yet have a way to configure session persistence. Acnodal EPIC Docs |
| Apache APISIX | Implementation (Nginx) | Cookie-Based | hash_on=[vars | header | cookie | consumer] key=cookie_name |
Upstream (Route or Backends) | N/A |
| Implementation (Nginx) | Header-Based | hash_on=[vars | header | cookie | consumer] key=header_name |
Upstream (Route or Backends) | N/A | |
| Apache httpd | Web Server | Cookie-Based / URL-Encoded | Cookie Attributes | N/A | N/A |
| Cilium | Implementation / Dataplane | None | None | None | Cilium has no documented way of doing session persistence. Cilium Docs |
| Contour | Implementation (Envoy) | Cookie-Based | Name=name pathRewrite=path domainRewrite=domain secure sameSite |
Route and Service (Backends) | Envoy does not natively support cookie attribute rewriting nor adding attributes other than path and TTL, but rewriting and adding additional attributes is possible via Lua (Contour design reference, Envoy Issue). |
| Emissary-Ingress | Implementation (Envoy) | Cookie-Based | Name=name Path=path TTL=duration |
Module or Mapping (Global or Route) | N/A |
| Header-Based | Name=name | Module or Mapping (Global or Route) | N/A | ||
| Envoy | Dataplane | Cookie-Based | Name=name Path=path TTL=duration |
HttpConnectionManager (Route) | Envoy does not natively support cookie attribute rewriting nor adding attributes other than path and TTL, but rewriting and adding additional attributes is possible via Lua (Contour design reference, Envoy Issue). |
| Header-Based | Name=name | HttpConnectionManager (Route) | N/A | ||
| Envoy Gateway | Implementation (Envoy) | N/A | Supports Gateway API Only* | N/A | *Envoy Gateway solely uses Gateway API; therefore, it doesn’t yet have a way to configure session persistence. Envoy Gateway Docs |
| Flomesh Service Mesh | Implementation / Dataplane (Pipy) | ? | ? | ? | ? |
| Gloo Edge 2.0 | Implementation (Envoy) | Cookie-Based | Name=name Path=path TTL=duration |
Route (Route) | N/A |
| Header-Based | Name=name | Route (Route) | N/A | ||
| Google CloudRun | Implementation / Dataplane | Cookie-Based | Enabled / Disabled | Service (Backends) | Only allowed to turn off or on, no other configuration items |
| Google Kubernetes Engine | Implementation / Dataplane | Cookie-Based | GENERATED_COOKIE or HTTP_COOKIE=name cookieTtlSec |
Backend Policy (Backends) | Google Kubernetes Engine lists the products that can do persistence/affinity mode. All persistence/affinity options are exclusive and can’t be used at the same time. Note: Google Kubernetes Engine defines everything (persistence and affinity) as session affinity. |
| Header-Based | httpHeaderName=name | Backend Policy (Backends) | N/A | ||
| HAProxy | Dataplane | Cookie-Based | name=name [rewrite | insert | prefix ] indirect nocache postonly preserve httponly secure domain=domain maxidle=idle maxlife=life dynamic attr=value |
Default or Backends (Global or Backends) | HAProxy allows for operational cookie strategy configuration (i.e. when/how HAProxy should inject cookies) |
| HAProxy Ingress | Implementation (HAProxy) | Cookie-Based | affinity (enable/disable) cookie-key=key session-cookie-domain=domain session-cookie-dynamic=[true | false] session-cookie-keywords=keywords session-cookie-name=name session-cookie-preserve=[true | false] session-cookie-same-site=[true | false] session-cookie-shared=[true | false] session-cookie-strategy=strategy session-cookie-value-strategy=value_strategy |
Backend (Backends) | N/A |
| Hashicorp Consul | Implementation (Envoy) | N/A | Supports Gateway API Only* | N/A | *Hashicorp Consul solely uses Gateway API; therefore, it doesn’t yet have a way to configure session persistence. Hashicorp Consul API Gateway Docs |
| Istio | Implementation (Envoy) | Cookie-Based | Name=name Path=path TTL=duration |
ConsistentHashLB (Backends) | Istio also supports turning on cookie-based session persistence via the Pilot ENV variable PILOT_PERSISTENT_SESSION_LABEL. |
| Implementation (Envoy) | Header-Based | Name=name | ConsistentHashLB (Backends) | N/A | |
| Java Servlet | Web Server | Cookie-Based / URL-Encoded | invalidate() setAttribute(String name, Object value) setMaxInactiveInterval(int interval) |
N/A | Java Servlets do not natively support proxy functions. |
| Kong | Implementation / Dataplane | Cookie-Based | cookie_name=name rolling_timeout=timeout absolute_timeout=timeout idling_timeout=timeout cookie_path=path cookie_domain=domain cookie_same_site=[Strict | Lax | None | off] cookie_http_only cookie_secure=[true | false] stale_ttl=duration cookie_persistent=[true | false] storage=storage_type |
Route, Service, Global (Route or Backends or Global) | N/A |
| Header-Based | name | Upstreams (Backends) | N/A | ||
| Kuma | Implementation (Envoy) | None | None | None | Kuma has no documentation on how it supports session persistence or cookies. Kuma Docs |
| Nginx | Dataplane | Cookie-Based (Nginx Plus Only) | Name=name Expires=time Domain=domain HttpOnly SameSite = [strict | lax | none | $variable] Secure path=path |
Upstream (Backends) | See also Sticky Cookie |
| NGINX Kubernetes Gateway | Implementation (Nginx) | N/A | Supports Gateway API Only* | N/A | *NGINX Kubernetes Gateway solely uses Gateway API; therefore, it doesn’t yet have a way to configure session persistence. Nginx Kubernetes Gateway Docs |
| Traefik | Implementation / Dataplane | Cookie-Based | name=name secure httpOnly sameSite=[none | lax | strict ] |
Services (Backends) | N/A |
Session Affinity
| Technology | Technology Type | Session Affinity Type | Notes |
|---|---|---|---|
| Acnodal EPIC | Implementation (Envoy) | Supports Gateway API Only* | *Acnodal Epic solely uses Gateway API; therefore, it doesn’t yet have a way to configure session affinity. Acnodal EPIC Docs |
| Apache APISIX | Implementation (Nginx) | Consistent Hashing via consumer ID or other variables | N/A |
| Cilium | Implementation / Dataplane | Source IP Address (Default) | N/A |
| Contour | Implementation (Envoy) | Consistent Hashing via RequestHash | N/A |
| Emissary-Ingress | Implementation (Envoy) | Consistent Hashing via sourceIP | N/A |
| Envoy | Dataplane | Consistent Hashing via Maglev and Ring Hash | N/A |
| Envoy Gateway | Implementation (Envoy) | Supports Gateway API Only* | *Envoy Gateway solely uses Gateway API; therefore, it doesn’t yet have a way to configure session affinity. Envoy Gateway Docs |
| Flomesh Service Mesh | Implementation / Dataplane (Pipy) | ? | ? |
| Gloo Edge 2.0 | Implementation (Envoy) | Consistent Hashing via HashPolicy of SourceIP | N/A |
| Google CloudRun | Implementation / Dataplane | None | Google CloudRun Docs |
| Google Kubernetes Engine | Implementation / Dataplane | CLIENT_IP_NO_DESTINATION and CLIENT_IP | Google Kubernetes Engine lists the products that can do persistence/affinity mode. Session affinity settings are fulfilled only if the load balancing locality policy (LocalityLbPolicy) is set to RING_HASH or MAGLEV. All persistence/affinity options are exclusive and can’t be used at the same time. Note: Google Kubernetes Engine defines everything (persistence and affinity) as session affinity. |
| HAProxy | Dataplane | Stick Tables using connection attributes | N/A |
| HAProxy Ingress | Implementation (HAProxy) | None | Haproxy Ingress Docs |
| Hashicorp Consul | Implementation (Envoy) | Supports Gateway API Only* | *Hashicorp Consul solely uses Gateway API; therefore, it doesn’t yet have a way to configure session affinity. Hashicorp Consul API Gateway Docs |
| Istio | Implementation (Envoy) | Consistent Hashing via ConsistentHashLB with Maglev or Ring Hash | N/A |
| Kong | Implementation / Dataplane | Consistent Hashing via Source IP or Consumer ID | N/A |
| Kuma | Implementation (Envoy) | Consistent Hashing via Ring Hash | N/A |
| Nginx | Dataplane | Consistent Hashing via IP or other connection attributes (Nginx Open Source) | N/A |
| NGINX Kubernetes Gateway | Implementation (Nginx) | Supports Gateway API Only* | *NGINX Kubernetes Gateway solely uses Gateway API; therefore, it doesn’t yet have a way to configure session affinity. Nginx Kubernetes Gateway Docs |
| Traefik | Implementation / Dataplane | None | Traefik Docs |
Sessions in Java¶
Java application servers such as Tomcat and Jetty, were the first to standardize the API around persistent sessions. These Java applications introduced the “jsessionid” cookie as well as more advanced features such as session migration, replication, and on demand session activation. It’s important for Gateway API to examine persistent session use cases and history from Java APIs to ensure the API is designed appropriately.
API¶
Session Persistence API¶
The majority of implementations and dataplanes support configuring cookies for session persistence while support for header-based session persistence is limited.
To organize potential Gateway API configuration of cookie functionality, the next section will break down configuration into specific categories: - Session Cookie Configuration - Session Cookie Strategy
Session Cookie Configuration
Session cookie configuration are the parameters and attributes the dataplane uses to create a cookie for establishing a session.
A set-cookie request can be broken down into 3 main components: - Cookie Name - Cookie Value - Cookie Attributes
Only the Cookie Name and Cookie Value are required. All Cookie Attributes are optional. Attributes are simply key=value pairs with the value as optional for some attributes.
The set-cookies attributes defined by rfc6265 are: - Expires=date - Max-Age=number - Domain=domain - Path=path-value - Secure - HttpOnly
However, there are de-facto standard attributes that have been proposed as updates to rfc6265: - SameSite=[Strict|Lax|None] (Draft RFC Update) - Partitioned (Draft RFC Update)
Session Cookie Strategy
Session cookie strategy is configuration specific to the creation of the cookie by the dataplane. These are not specific configuration parameters of the cookie itself, but instead the operation to take when inserting the cookie such as: - Rewriting an existing cookie value - Inserting a new cookie - Prefixing an existing cookie - Preserving an existing cookie value - Only adding cookies on specific HTTP methods (POST/GET)
Session Affinity API¶
TBD - See open question about session affinity API
Open Questions¶
- Should we include both session persistence and session affinity API configurations in this GEP or just focus on session persistence?
- Should both session persistence and session affinity be designed as common load balancing policy API? Should they be configured as separately in the API?
- What happens when session persistence is broken because the backend is not up or healthy? If that's an error case, how should that be handled? Should the API dictate the http error code? Or should the API dictate fall back behavior?
TODO¶
The following are items that we intend to resolve before we consider this GEP implementable:
- We need to identify the needs and use cases from end users for specifying session persistence.
- We need to identify and document requirements regarding session draining and migration. How do implementations drain established sessions during backend upgrades without disruption?
- We need to document sessions with Java in greater detail. Java standardized the API and behavior of session persistence long ago and would be worth examining.
- We need to add a small section on compliance regarding the browser and client relationship.
- We need to design and document an API for session persistence and/or session affinity in the Gateway API spec (see open question about API design)
Alternatives¶
Alternate Naming¶
This GEP describes session persistence and session affinity as the idea of strong and weak connection persistence respectively. Other technologies use different names or define persistence and affinity differently:
- Envoy defines stateful sessions as what we've defined as session persistence
- Google Cloud defines session affinity as what we've defined as session persistence
- Nginx defines session persistence as what we've defined as both session persistence and affinity
- Traefik defines sticky sessions as what we've defined as session persistence
- Apache httpd defines sticky sessions or stickyness as what we've defined as session persistence
- Kubernetes defines session affinity based on client IP hashing (same as our session affinity)
Though session persistence is a ubiquitous name, session affinity is more inconsistently used. An alternate decision could be made to use a different name for session affinity based on the prevalence of other naming conventions.
References¶
- LBPolicy (proposed extension for session persistence API)
- gRPC Stateful Session Affinity Proposal (info on session draining and session persistence in gRPC)
- Kube-Proxy Session Affinity