Multi-Stage Transparent Server Load Balancing
Naoki Matsuhira
WIDE Project
Japan
naoki.matsuhira@gmail.com
4 April 2022
This document specifies Multi-Stage Transparent Server Load Balancing (MSLB) specification. MSLB make server load balancing over Layer3 network without packet header change at client and server. MSLB make server load balancing with any protocol and protocol with encription such as IPsec ESP, SSL/TLS.

Requirements Language
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119.

Introduction
This document specifies Multi-Stage Transparent Server Load Balancing (MSLB) specification.
MSLB provide server load balancing function over Layer3 network without packet header change at client and server. MSLB work with any protocol and protocol with payload encription such as IPsec ESP, SSL/TLS.

Traditional load balancing method
There are several load balancing technique, such as round robin DNS, IP Anycasting and destination address translation. Figure 1 shows load balancing system with typical server load balancer with destination address translation technique.

                                           +---------+   +--------+
                                           |         +---+ Server |
              +---------+   +----------+   |         |   +--------+
              |         |   |          |   |         |        :
 +--------+   |         |   |  Server  |   |         |   +--------+
 | Client +---+ Network +---+   Load   +---+ Network +---+ Server |
 +--------+   |         |   | Balancer |   |         |   +--------+
              |         |   |          |   |         |        :
              +---------+   +----------+   |         |   +--------+
                                           |         +---+ Server |
                                           +---------+   +--------+

It is well-known that Network address translator break internet transparency and have a application dependency characteristic.
Some server load balancer use application data, so with IPsec ESP, SSL/TLS, this mechanisms may not work well.

Architecture of MSLB
Load balancing is the tecnique that distribute packet to multiple server. For packet distribution, destination addresss translation technique is useful, however this technique itself break internet transparency.
After distribution, if write back to the original destination address may possoble, it is possible to recover transparency. This is the basic idea and architecture of MSLB. Figure 2 shows architecture of MSLB.

  Client ----  overwrite    +----------  write back  ----- server
               destination  |
               address      + ---------  write back  ----- server
                            |
                            :                 :              :

                            + ---------  write back  ----- server

This method process only destination address of IP header. This method can be applied to both IPv4 and IPv6.

configuration

basic configuration
Figure 3 shows basic server load balancing system with MSLB. This case two-stage configuration with one MSLB-F and one-stage many MSLB-Bs.

                                   +-------+   +------+   +------+
                                   |       +---+MSLB-B+---+Server|
            +-------+   +------+   |       |   +------+   +------+
            |       |   |      |   |       |       :          :
 +------+   |       |   |      |   |       |   +------+   +------+
 |Client+---+Network+---+MSLB-F+---+Network+---+MSLB-B+---+Server|
 +------+   |       |   |      |   |       |   +------+   +------+
            |       |   |      |   |       |       :          :
            +-------+   +------+   |       |   +------+   +------+
                                   |       +---+MSLB-B+---+Server|
                                   +-------+   +------+   +------+

MSLB-F is front function of MSLB and translate destination address to one of the address of MSLB-B. BSLB-B s backend function of MSLB and translate destination address to the original server address, i.e. address of MSLB-F. The IP address of MSLB-F and all server is the same value.
MSLB-F may multi-stage configuration. Figure 4shows three stage configuration with two-stage MSLB-F and one-stage many MSLB-Bs.

                                         +---+  +------+  +------+
                                         |   |--+MSLB-B+--+Server|
                          +---+          |   |  +------+  +------+
                          |   |  +----+  |Net|      :         :
           +---+  +----+  |   |  |MSLB|  |   |  +------+  +------+
           |   |  |    |  |   |--+ -F +--+   |--+MSLB-B+--+Server|
 +------+  |   |  |    |  |   |  +----+  +---+  +------+  +------+
 |Client+--+Net+--+MSLB+--+Net|
 +------+  |   |  | -F |  |   |  +----+  +---+  +------+  +------+
           |   |  |    |  |   +--+MSLB+--+   |--+MSLB-B+--+Server|
           +---+  +----+  |   |  | -F |  |   |  +------+  +------+
                          |   |  +----+  |Net|      :         :
                          +---+          |   |  +------+  +------+
                                         |   |--+MSLB-B+--+Server|
                                         +---*  +------+  +------+

one arm configuration
Figure 5shows one arm configuration of server load balancing system with MSLB.

                +---------+
                |         |
                | MSLB-F  |
                |         |
                +----+----+
                     |
                +----+----+   +--------+   +--------+
                |         +---+ MSLB-B +---+ Server |
                |         |   +--------+   +--------+
                |         |       :            :
 +--------+     |         |   +--------+   +--------+
 | Client |-----+ Network +---+ MSLB-B +---+ Server |
 +--------+     |         |   +--------+   +--------+
                |         |       :            :
                |         |   +--------+   +--------+
                |         +---+ MSLB-B +---+ Server |
                +---------+   +--------+   +--------+

MSLB-F is front function of MSLB and translate destination address to one of the address of MSLB-B. BSLB-B s backend function of MSLB and translate destination address to the original server address, i.e. address of MSLB-F. The IP address of MSLB-F and all server is the same value.
This configuration, MSLB-F is connecting to the network with single link, that is one arm configuration. This case, retuen packet, i.e. packet from server to client does not pass through the MSLB-F.

mode
MSLB have two mode, one is address translation mode, and the other is encapsulation mode.

address translation mode
This mode using address translation technique.
Figure 6 shows packet processing with address translation mode.

                                   +-------+   +------+   +------+
                                   |       +---+MSLB-B+---+Server|
 +------+                          |       |   | IP_B1|   | IP_S |
 |Client|   +-------+   +------+   |       |   +------+   +------+
 | IP_C1+---+       |   |      |   |       |
 +------+   |       |   |      |   |       |   +------+   +------+
            |Network|   |MSLB-F|---+Network+---+MSLB-B+---+Server|
            |       +---+      |   |       |   | IP_B2|   | IP_S |
 +------+   |       |   | IP_S |   |       |   +------+   +------+
 |Client+---+       |   |      |   |       |
 | IP_C2|   +-------+   +------+   |       |   +------+   +------+
 +------+                          |       +---+MSLB-B+---+Server|
                                   |       |   | IP_B3|   | IP_S |
                                   +-------+   +------+   +------+
                           :                       :
                           :                       :
   +------+----+           :   +------+----+       :+------+----+
   | data | IP |           :   | data | IP |       :| data | IP |
   +------+----+           :   +------+----+       :+------+----+
   ----------------------> : --------------------> : ------------>
     src = IP_C1           :    src = IP_C1        :   src = IP_C1
     dst = IP_S            :    dst = IP_B1        :   dst = IP_S
                           :                       :
   +------+----+           :   +------+----+       :+------+----+
   | data | IP |           :   | data | IP |       :| data | IP |
   +------+----+           :   +------+----+       :+------+----+
   <--------------------- -: <-------------------- : <------------
     src = IP_S            :    src = IP_S         :   src = IP_S
     dst = IP_C1           :    dst = IP_C1        :   dst = IP_C1
                           :                       :

In this figure, to the Client, IP address is allocated IP_C1, IP_C2, and server IP address is IP_S. This case, IP_S is also allocate to all servers and MSLB-F. And to the MSLB-B, IP_B1, IP_B2, IP_B3 is allocated. These allocation is shown in upper part of Figure 6.
Lower part of Figure 6 shows packet transfered between client and server. From Client to the Server, only destination address is translate, MSLB-F translate from IP_S to IP_B1, and MSLB-B translate from IP_B1 to IP_S. Then the destination address of packet which send client and the destination address of packet which recieve server is same address. That mean, transparency is remained.
Return packet, i.e., from server to the client is not translate, just forwarded.
In the Internet, Client IP address and server IP address must Global IP address, however, IP address of MSLB-B may private IP address.

 +--------------------+----------+-------------------------+
 | Source IP address  | net mask | destination IP address  |
 +--------------------+----------+-------------------------+
 |  IP_C1             |          |  IP_B1                  |
 +--------------------+----------+-------------------------+
 |  IP_C2             |          |  IP_B2                  |
 +--------------------+----------+-------------------------+
 |        :           |    :     |             :           |
 |        :           |    :     |             :           |
 |        :           |    :     |             :           |
 +--------------------+----------+-------------------------+

Figure 7 shows MSLB table. MSLB have this table and translate the destination address using this table value. MSLB-F check source IP address, and translate destination address with this table.
Using IPv4-IPv6 translation may possible, i.e., IPv4 packet translated to IPv6, then translate to IPv4 or IPv6 packet translate to IPv4, then translate IPv6 may possibleFigure 8 shows possible combination of IPv4 and IPv6. These IPv4-IPv6 translation case will be defined in future.

     Client     MSLB-F         MSLB-B     Server
                  :              :
                  :              :
(1)  <-- IPv4 --> : <-- IPv4 --> : <-- IPv4 -->
                  :              :
(2)  <-- IPv6 --> : <-- IPv6 --> : <-- IPv6 -->
                  :              :
(3)  <-- IPv4 --> : <-- IPv6 --> : <-- IPv4 -->
                  :              :
(4)  <-- IPv6 --> : <-- IPv4 --> : <-- IPv6 -->
                  :              :

encapsulation mode
This mode using encapsulation technique.
Figure 9 shows packet processing with encapsulation mode.

                                   +-------+   +------+   +------+
                                   |       +---+MSLB-B+---+Server|
                                   |       |   | IP_B1|   | IP_S |
            +-------+   +------+   |       |   +------+   +------+
            |       |   |      |   |       |
 +------+   |       |   |      |   |       |   +------+   +------+
 |Client|---+Network+---+MSLB-F|---+Network+---+MSLB-B+---+Server|
 | IP_C |   |       |   |      |   |       |   | IP_B2|   | IP_S |
 +------+   |       |   | IP_S |   |       |   +------+   +------+
            |       |   |      |   |       |
            +-------+   +------+   |       |   +------+   +------+
                                   |       +---+MSLB-B+---+Server|
                                   |       |   | IP_B3|   | IP_S |
                                   +-------+   +------+   +------+
                           :                       :
                           :                       :
   +------+----+           :   +------+----+----+  :+------+----+
   | data | IP |           :   | data | IP | IP |  :| data | IP |
   +------+----+           :   +------+----+----+  :+------+----+
   ----------------------> : --------------------> : ------------>
     src = IP_C            :   Inner header        :   src = IP_C
     dst = IP_S            :    src = IP_C         :   dst = IP_S
                           :    dst = IP_S         :
                           :   Outer header        :
                           :    src = IP_S         :
                           :    dst = IP_B1        :
                           :                       :
                           :                       :
                           :                       :
   +------+----+           :   +------+----+       :+------+----+
   | data | IP |           :   | data | IP |       :| data | IP |
   +------+----+           :   +------+----+       :+------+----+
   <--------------------- -: <-------------------- : <------------
     src = IP_S            :    src = IP_S         :   src = IP_S
     dst = IP_C            :    dst = IP_C         :   dst = IP_C
                           :                       :

In this figure, to the Client, IP address is allocated IP_C1, IP_C2, and server IP address is IP_S. This case, IP_S is also allocate to all servers and MSLB-F. And to the MSLB-B, IP_B1, IP_B2, IP_B3 is allocated. These allocation is shown in upper part of Figure 6.
Lower part of Figure 6 shows packet transfered between client and server. From Client to the Server, MSLB-F encapsulate original IP packet and send to MSLB-B. MSLB-B decapsulate outer IP header, and forwarad to the server. Inner IP packet does not change, that mean, transparency is remained.
With encapsulation mode, packet size is increase, so fragmentation is needed if encapsulated packet size exceed MTU or Path MTU. MSLB-F MUST support tunnel MTU discovery. Fragmentation and Path MTU discovery issue will describe in future.
Return packet, i.e., from server to the client is not encapsulate, just forwarded.
In the Internet, Client IP address and server IP address must Global IP address, however, IP address of MSLB-B may private IP address.

 +--------------------+----------+-------------------------+
 | Source IP address  | net mask | destination IP address  |
 +--------------------+----------+-------------------------+
 |  IP_C1             |          |  IP_B1                  |
 +--------------------+----------+-------------------------+
 |  IP_C2             |          |  IP_B2                  |
 +--------------------+----------+-------------------------+
 |        :           |    :     |             :           |
 |        :           |    :     |             :           |
 |        :           |    :     |             :           |
 +--------------------+----------+-------------------------+

Figure 10 shows MSLB table. MSLB have this table and encapsulate and generate outer header with destination address using this table value. MSLB-F check source IP address, and generate destination address of outer header with this table.
Using IPv4 over IPv6 encapsulation or IPv6 over IPv4 encapsulation may possible, i.e., IPv4 packet encapsulated to IPv6, then decapsulate to IPv4 or IPv6 packet encapsulated to IPv4, then deencapsulated IPv6 may possibleFigure 11 shows possible combination of IPv4 and IPv6. These IPv4-IPv6 encapsulation case will be defined in future.

     Client     MSLB-F                   MSLB-B     Server
                  :                        :
                  :                        :
(1)  <-- IPv4 --> : <-- IPv4 over IPv4 --> : <-- IPv4 -->
                  :                        :
(2)  <-- IPv6 --> : <-- IPv6 over IPv6 --> : <-- IPv6 -->
                  :                        :
(3)  <-- IPv4 --> : <-- IPv4 over IPv6 --> : <-- IPv4 -->
                  :                        :
(4)  <-- IPv6 --> : <-- IPv6 over IPv4 --> : <-- IPv6 -->
                  :                        :

Ingress filtering environment
[RFC2827] describe ingress filtering for defending DoS attack which employ IP source address spoofing.
Depend on the location of the MSLB-F and MSLB-B, it is possible that packet from server to client is discarded by ingress filtering. In such case, encapsulating the packet from server to client might resolve. Figure 12 shows such solution.

                                   +-------+   +------+   +------+
                                   |       +---+MSLB-B+---+Server|
 +------+                          |       |   | IP_B1|   | IP_S |
 |Client|   +-------+   +------+   |       |   +------+   +------+
 | IP_C1+---+       |   |      |   |       |
 +------+   |       |   |      |   |       |   +------+   +------+
            |Network|   |MSLB-F|---+Network+---+MSLB-B+---+Server|
            |       +---+      |   |       |   | IP_B2|   | IP_S |
 +------+   |       |   | IP_S |   |       |   +------+   +------+
 |Client+---+       |   |      |   |       |
 | IP_C2|   +-------+   +------+   |       |   +------+   +------+
 +------+                          |       +---+MSLB-B+---+Server|
                                   |       |   | IP_B3|   | IP_S |
                                   +-------+   +------+   +------+

                           :                       :
   +------+----+           :   +------+----+----+  :+------+----+
   | data | IP |           :   | data | IP | IP |  :| data | IP |
   +------+----+           :   +------+----+----+  :+------+----+
   <--------------------- -: <-------------------- : <------------
     src = IP_S            :   Inner header        :   src = IP_S
     dst = IP_C            :    src = IP_S         :   dst = IP_C
                           :    dst = IP_C         :
                           : Outer header          :
                           :    src = IP_B1
                           :    dst = IP_TBD

Characteristic
MSLB has following characteristics.
- Layer 3 Load balancer
- Support NAT unfriendly application such as FTP
- work with any application layer protocol (maybe)
- work with encription (IPsec ESP, SSL/TLS)
- work over Layer 3 network
- may enforce policy with static configuration

IANA Considerations
This document makes no request of IANA.
Note to RFC Editor: this section may be removed on publication as an RFC.

Security Considerations
Security consideration does not discussed in this memo.

Acknowledgements

Normative References
[RFC2827]