Checksum Offloads

Introduction

This document describes a set of techniques in the Linux networking stack to take advantage of checksum offload capabilities of various NICs.

The following technologies are described:

  • TX Checksum Offload

  • LCO: Local Checksum Offload

  • RCO: Remote Checksum Offload

Things that should be documented here but aren’t yet:

  • CHECKSUM_UNNECESSARY conversion

TX Checksum Offload

In brief, Tx checksum offload allows to request the device fill in a single ones-complement checksum defined by the sk_buff fields skb->csum_start and skb->csum_offset. The device should compute the 16-bit ones-complement checksum (i.e. the ‘IP-style’ checksum) from csum_start to the end of the packet, and fill in the result at (csum_start + csum_offset).

Because csum_offset cannot be negative, this ensures that the previous value of the checksum field is included in the checksum computation, thus it can be used to supply any needed corrections to the checksum (such as the sum of the pseudo-header for UDP or TCP).

This interface only allows a single checksum to be offloaded. Where encapsulation is used, the packet may have multiple checksum fields in different header layers, and the rest will have to be handled by another mechanism such as LCO or RCO.

SCTP CRC32c can also be offloaded using this interface, by means of filling skb->csum_start and skb->csum_offset as described above, setting skb->csum_not_inet, and advertising NETIF_F_SCTP_CRC. Drivers must not treat ordinary IP checksum offload as SCTP CRC32c support.

No offloading of the IP header checksum is performed; it is always done in software. This is OK because when we build the IP header, we obviously have it in cache, so summing it isn’t expensive. It’s also rather short.

The requirements for GSO are more complicated, because when segmenting an encapsulated packet both the inner and outer checksums may need to be edited or recomputed for each resulting segment.

A driver declares its offload capabilities in netdev->hw_features; see Netdev features mess and how to get out from it alive for more. NETIF_F_IP_CSUM and NETIF_F_IPV6_CSUM are restricted legacy features and are being deprecated in favor of NETIF_F_HW_CSUM. New devices should use NETIF_F_HW_CSUM to advertise generic checksum offload. The skb_csum_hwoffload_help() helper can resolve CHECKSUM_PARTIAL according to the device’s advertised checksum capabilities, falling back to software when needed.

The stack should, for the most part, assume that checksum offload is supported by the underlying device. The only place that should check is validate_xmit_skb(), and the functions it calls directly or indirectly. That function compares the offload features requested by the SKB (which may include other offloads besides TX Checksum Offload) and, if they are not supported or enabled on the device (determined by netdev->features), performs the corresponding offload in software. In the case of TX Checksum Offload, that means calling skb_csum_hwoffload_help(skb, features).

LCO: Local Checksum Offload

LCO is a technique for efficiently computing the outer checksum of an encapsulated datagram when the inner checksum is due to be offloaded.

The ones-complement sum of a correctly checksummed TCP or UDP packet is equal to the complement of the sum of the pseudo header, because everything else gets ‘cancelled out’ by the checksum field. This is because the sum was complemented before being written to the checksum field.

More generally, this holds in any case where the ‘IP-style’ ones complement checksum is used, and thus any checksum that TX Checksum Offload supports.

That is, if we have set up TX Checksum Offload with a start/offset pair, we know that after the device has filled in that checksum, the ones complement sum from csum_start to the end of the packet will be equal to the complement of whatever value we put in the checksum field beforehand. This allows us to compute the outer checksum without looking at the payload: we simply stop summing when we get to csum_start, then add the complement of the 16-bit word at (csum_start + csum_offset).

Then, when the true inner checksum is filled in (either by hardware or by skb_checksum_help()), the outer checksum will become correct by virtue of the arithmetic.

LCO is performed by the stack when constructing an outer UDP header for an encapsulation such as VXLAN or GENEVE, in udp_set_csum(). Similarly for the IPv6 equivalents, in udp6_set_csum().

It is also performed when constructing GRE headers with the shared gre_build_header() helper in include/net/gre.h, which is used by both IPv4 and IPv6 GRE.

All of the LCO implementations use a helper function lco_csum(), in include/linux/skbuff.h.

LCO can safely be used for nested encapsulations; in this case, the outer encapsulation layer will sum over both its own header and the ‘middle’ header. This does mean that the ‘middle’ header will get summed multiple times, but there doesn’t seem to be a way to avoid that without incurring bigger costs (e.g. in SKB bloat).

RCO: Remote Checksum Offload

RCO is a technique for eliding the inner checksum of an encapsulated datagram, allowing the outer checksum to be offloaded. It does, however, involve a change to the encapsulation protocols, which the receiver must also support. For this reason, it is disabled by default.

RCO is detailed in the following Internet-Drafts:

In Linux, RCO is implemented individually in each encapsulation protocol, and most tunnel types have flags controlling its use. For instance, VXLAN has the configuration flag VXLAN_F_REMCSUM_TX to indicate that RCO should be used when transmitting.

RX Checksum Offload

RX checksum offload is controlled via NETIF_F_RXCSUM. When disabled the driver must not set skb->ip_summed on ingress packets. As mentioned, IPv4 checksum is not offloaded, the RXCSUM feature controls the offload of verification of transport layer checksums.

Note that packets with bad TCP/UDP checksums must still be passed to the stack. skb->ip_summed of such packets can be set to CHECKSUM_COMPLETE or left at CHECKSUM_NONE. Drivers must not discard packets with bad TCP/UDP checksum and must not configure the device to drop them. Checksum validation is relatively inexpensive and having bad packets reflected in SNMP counters is crucial for network monitoring.

skb checksum documentation

The interface for checksum offload between the stack and networking drivers is as follows...

Checksumming of received packets by device

Indication of checksum verification is set in sk_buff.ip_summed. Possible values are:

  • CHECKSUM_NONE

    Device did not checksum this packet e.g. due to lack of capabilities. The packet contains full (though not verified) checksum in packet but not in skb->csum. Thus, skb->csum is undefined in this case.

  • CHECKSUM_UNNECESSARY

    The hardware you’re dealing with doesn’t calculate the full checksum (as in CHECKSUM_COMPLETE), but it does parse headers and verify checksums for specific protocols. For such packets it will set CHECKSUM_UNNECESSARY if their checksums are okay. sk_buff.csum is still undefined in this case though. A driver or device must never modify the checksum field in the packet even if checksum is verified.

    CHECKSUM_UNNECESSARY is applicable to following protocols:

    • TCP: IPv6 and IPv4.

    • UDP: IPv4 and IPv6. A device may apply CHECKSUM_UNNECESSARY to a zero UDP checksum for either IPv4 or IPv6, the networking stack may perform further validation in this case.

    • GRE: only if the checksum is present in the header.

    • SCTP: indicates the CRC in SCTP header has been validated.

    • FCOE: indicates the CRC in FC frame has been validated.

    sk_buff.csum_level indicates the number of consecutive checksums found in the packet minus one that have been verified as CHECKSUM_UNNECESSARY. For instance if a device receives an IPv6->UDP->GRE->IPv4->TCP packet and a device is able to verify the checksums for UDP (possibly zero), GRE (checksum flag is set) and TCP, sk_buff.csum_level would be set to two. If the device were only able to verify the UDP checksum and not GRE, either because it doesn’t support GRE checksum or because GRE checksum is bad, skb->csum_level would be set to zero (TCP checksum is not considered in this case).

  • CHECKSUM_COMPLETE

    This is the most generic way. The device supplied checksum of the _whole_ packet as seen by netif_rx() and fills in sk_buff.csum. This means the hardware doesn’t need to parse L3/L4 headers to implement this.

    Notes:

    • Even if device supports only some protocols, but is able to produce skb->csum, it MUST use CHECKSUM_COMPLETE, not CHECKSUM_UNNECESSARY.

    • CHECKSUM_COMPLETE is not applicable to SCTP and FCoE protocols.

  • CHECKSUM_PARTIAL

    A checksum is set up to be offloaded to a device as described in the output description for CHECKSUM_PARTIAL. This may occur on a packet received directly from another Linux OS, e.g., a virtualized Linux kernel on the same host, or it may be set in the input path in GRO or remote checksum offload. For the purposes of checksum verification, the checksum referred to by skb->csum_start + skb->csum_offset and any preceding checksums in the packet are considered verified. Any checksums in the packet that are after the checksum being offloaded are not considered to be verified.

Checksumming on transmit for non-GSO

The stack requests checksum offload in the sk_buff.ip_summed for a packet. Values are:

  • CHECKSUM_PARTIAL

    The driver is required to checksum the packet as seen by hard_start_xmit() from sk_buff.csum_start up to the end, and to record/write the checksum at offset sk_buff.csum_start + sk_buff.csum_offset. A driver may verify that the csum_start and csum_offset values are valid values given the length and offset of the packet, but it should not attempt to validate that the checksum refers to a legitimate transport layer checksum -- it is the purview of the stack to validate that csum_start and csum_offset are set correctly.

    When the stack requests checksum offload for a packet, the driver MUST ensure that the checksum is set correctly. A driver can either offload the checksum calculation to the device, or call skb_checksum_help (in the case that the device does not support offload for a particular checksum).

    NETIF_F_IP_CSUM and NETIF_F_IPV6_CSUM are being deprecated in favor of NETIF_F_HW_CSUM. New devices should use NETIF_F_HW_CSUM to indicate checksum offload capability. skb_csum_hwoffload_help() can be called to resolve CHECKSUM_PARTIAL based on network device checksumming capabilities: if a packet does not match them, skb_checksum_help() or skb_crc32c_help() (depending on the value of sk_buff.csum_not_inet, see Non-IP checksum (CRC) offloads) is called to resolve the checksum.

  • CHECKSUM_NONE

    The skb was already checksummed by the protocol, or a checksum is not required.

  • CHECKSUM_UNNECESSARY

    This has the same meaning as CHECKSUM_NONE for checksum offload on output.

  • CHECKSUM_COMPLETE

    Not used in checksum output. If a driver observes a packet with this value set in skbuff, it should treat the packet as if CHECKSUM_NONE were set.

Non-IP checksum (CRC) offloads

NETIF_F_SCTP_CRC

This feature indicates that a device is capable of offloading the SCTP CRC in a packet. To perform this offload the stack will set csum_start and csum_offset accordingly, set ip_summed to CHECKSUM_PARTIAL and set csum_not_inet to 1, to provide an indication in the skbuff that the CHECKSUM_PARTIAL refers to CRC32c. A driver that supports both IP checksum offload and SCTP CRC32c offload must verify which offload is configured for a packet by testing the value of sk_buff.csum_not_inet; skb_crc32c_csum_help() is provided to resolve CHECKSUM_PARTIAL on skbs where csum_not_inet is set to 1.

NETIF_F_FCOE_CRC

This feature indicates that a device is capable of offloading the FCOE CRC in a packet. To perform this offload the stack will set ip_summed to CHECKSUM_PARTIAL and set csum_start and csum_offset accordingly. Note that there is no indication in the skbuff that the CHECKSUM_PARTIAL refers to an FCOE checksum, so a driver that supports both IP checksum offload and FCOE CRC offload must verify which offload is configured for a packet, presumably by inspecting packet headers.

Checksumming on output with GSO

In the case of a GSO packet (skb_is_gso() is true), checksum offload is implied by the SKB_GSO_* flags in gso_type. Most obviously, if the gso_type is SKB_GSO_TCPV4 or SKB_GSO_TCPV6, TCP checksum offload as part of the GSO operation is implied. If a checksum is being offloaded with GSO then ip_summed is CHECKSUM_PARTIAL, and both csum_start and csum_offset are set to refer to the outermost checksum being offloaded (two offloaded checksums are possible with UDP encapsulation).