Encapsulating Security Payload (ESP) is a member of the Internet Protocol Security (IPsec) set of protocols that encrypt and authenticate the packets of data between computers using a Virtual Private Network (VPN). The focus and layer on which ESP operates makes it possible for VPNs to function securely.
The enhanced version of IPsec in use is an Internet-layer security protocol. It is pre-programmed for IP-layer application security whereas other protocols such as Transport Layer Security (TLS) and Secure Shell (SSH) function on the application layer.
Security Authentication Header (AH) is another IPsec member protocol. ESP and AH can operate between hosts and between networks. The can also operate in two modes: the less-secure Transport Mode that encrypts the data packet, for use between two workstations that are running a VPN client; and Tunnel Mode, which is more secure. Tunnel Mode encrypts the whole packet including header info and source, and is used between networks.
Example:
“Security for a VPN involves IPsec, and with IPsec’s protocols of AH and ESP, the connection between a user and a network is secure. Going further ESP, on the application layer, can run in its more secure Tunnel Mode offering the most privacy.”