IPsec and IKEv2

IPsec and IKEv2

Introduction

IPsec protects IP packets between two hosts or gateways, hiding and authenticating their payloads with no change to the applications running on top. It is really two layers working together. The Internet Key Exchange protocol, IKEv2, negotiates algorithms, runs a Diffie-Hellman exchange, and authenticates both sides. The Encapsulating Security Payload, ESP, then uses the keys that IKE produced to encrypt and authenticate the actual traffic, packet by packet.

This guide builds both layers. The key-derivation formulas of IKEv2 are checked against 280 known-answer test vectors published by NIST for exactly this purpose, across five hash functions. A real 2048-bit Diffie-Hellman exchange computes the shared secret that those formulas consume. Pre-shared-key authentication proves each side holds the same secret, and AES-GCM protects both the IKE control channel and the ESP data channel that follows it.

Table of Contents

• History and Standards
• The Design in Brief
• The IKE Header and Payloads
• The 2048-bit Diffie-Hellman Exchange
• Key Derivation: SKEYSEED and prf+
• Authentication with a Pre-Shared Key
• The AES-GCM Encrypted Payload
• The Encapsulating Security Payload
• Interactive Visualizer
• A Worked Example
• Python Implementation
• Where It Stands
• Limitations
• Security Status
• FAQ
• References

History and Standards

IPsec and its first key-exchange protocol, IKEv1, date to November 1998, in RFC 2401 and RFC 2409. IKEv1 split negotiation into two phases and offered several authentication modes, one of which, aggressive mode with a pre-shared key, turned out to leak enough information for an offline dictionary attack against the key. The IETF replaced it with IKEv2, first published as RFC 4306 in December 2005, revised as RFC 5996, and consolidated into RFC 7296 in October 2014, the document this guide follows. In 2023, RFC 9395 deprecated IKEv1 and moved its three core documents, RFCs 2407, 2408, and 2409, to Historic status.

ESP followed the same path. The original RFC 2406, from November 1998, defined encryption and a separate integrity check. The current RFC 4303, from December 2005, adds support for combined-mode algorithms like AES-GCM, where one operation provides both at once. RFC 4106 and RFC 5282 specify exactly how AES-GCM plugs into ESP and into the IKEv2 Encrypted payload, and this guide implements both.

The Design in Brief

An IPsec connection goes through the same stages as the SSH handshake covered in the SSH guide, though the details differ throughout.

1. IKE_SA_INIT. Both sides exchange proposed algorithms, Diffie-Hellman public values, and nonces. From these, each side computes SKEYSEED and derives seven keys.
2. IKE_AUTH. Each side proves it holds the same pre-shared key, inside a payload encrypted with the keys from step 1. This step also negotiates the first Child SA, the pair of ESP security associations that will carry data.
3. Child SA keying material. A further key-derivation step produces the AES-GCM keys for the new Child SA, one for each direction.
4. ESP traffic. From here on, IP packets travel inside ESP packets, encrypted and authenticated with the Child SA keys.

This guide stops after one ESP packet. It does not implement the connection teardown, rekeying, or the Authentication Header, ESP’s integrity-only sibling.

The IKE Header and Payloads

Every IKE message starts with a 28-byte header (RFC 7296 section 3.1):

IKE SA Initiator's SPI (8 bytes)
IKE SA Responder's SPI (8 bytes)
Next Payload | Major/Minor Version | Exchange Type | Flags
Message ID (4 bytes)
Length (4 bytes)

The two SPIs together identify the IKE security association. The responder’s SPI is zero in the very first message, since the responder has not chosen one yet. The Exchange Type names the kind of message: 34 for IKE_SA_INIT, 35 for IKE_AUTH. The Flags byte carries one bit for “this is a response” and one for “the original initiator sent this”.

Payloads that follow the header share a four-byte generic header: a Next Payload type, a reserved byte, and a two-byte length. This is how a message chains an arbitrary list of payloads: each one names the type of the payload after it, and a zero means “no more payloads”. My code builds and reads this header, the Security Association payload (one proposal, naming AES-GCM, the PRF, and the Diffie-Hellman group), the Key Exchange payload, the Nonce payload, and the Identification and Authentication payloads that the IKE_AUTH exchange adds.

The 2048-bit Diffie-Hellman Exchange

IKEv2 Diffie-Hellman group 14, defined in RFC 3526, is a 2048-bit safe prime with generator 2. It is the same kind of classic modular exponentiation that Whitfield Diffie and Martin Hellman published in 1976, not the elliptic-curve variant that the X25519 guide and the SSH guide use. Each side picks a private exponent, computes g^private mod p, and sends that public value. The shared secret is peer_public^private mod p, which both sides compute to the same number by the commutativity of exponentiation.

RFC 7296 section 2.14 requires this shared secret, called g^ir, to be represented as a big-endian octet string the length of the modulus, 256 bytes for group 14, padded with leading zero bytes if the number itself is shorter. My code follows this exactly, and I checked the prime itself character for character against the hexadecimal value printed in RFC 3526 section 3.

Key Derivation: SKEYSEED and prf+

Section 2.14 defines a quantity called SKEYSEED, computed once per IKE SA, from which every other key is derived:

SKEYSEED = prf(Ni | Nr, g^ir)

prf is the negotiated pseudorandom function, keyed HMAC in every case this guide covers. Section 2.13 defines prf+, an iterated expansion that produces as much keying material as needed:

prf+(K, S) = T1 | T2 | T3 | ...
T1 = prf(K, S | 0x01)
T2 = prf(K, T1 | S | 0x02)
T3 = prf(K, T2 | S | 0x03)

Seven keys come out of one prf+ call on SKEYSEED: SK_d for deriving Child SA keys, SK_ai and SK_ar for integrity in each direction, SK_ei and SK_er for encryption in each direction, and SK_pi and SK_pr for the AUTH payload. When the negotiated cipher is AES-GCM, section 7.1 of RFC 5282 says SK_ai and SK_ar are zero bytes long, since the cipher authenticates on its own, and SK_ei/SK_er each carry a 4-byte salt appended after the AES key itself.

NIST’s Cryptographic Algorithm Validation Program publishes known-answer test vectors for exactly this construction, built for SP 800-135. They come as the IKEv2 KDF download on its component testing page. The file gives Ni, Nr, g^ir, the two SPIs, and the hash function, then the expected SKEYSEED and a prf+ expansion of a stated length, called DKM. It also gives the Child SA keying material, prf+(SK_d, Ni | Nr), and the IKE SA rekey formula, SKEYSEED = prf(SK_d_old, g^ir_new | Ni | Nr). My code reproduces all five quantities for all 280 published cases, across SHA-1, SHA-224, SHA-256, SHA-384, and SHA-512, and two different nonce lengths.

Authentication with a Pre-Shared Key

Section 2.15 of RFC 7296 defines how each side proves it holds the authentication secret, without ever sending that secret itself. Each side signs a block of data built from the first two messages, the other side’s nonce, and its own identity:

InitiatorSignedOctets = RealMessage1 | NonceRData | MACedIDForI
MACedIDForI = prf(SK_pi, RestOfInitIDPayload)

With a pre-shared key, the “signature” is an HMAC chain:

AUTH = prf(prf(SharedSecret, "Key Pad for IKEv2"), SignedOctets)

The inner prf call turns the shared secret into a fixed-size key, so the outer HMAC never touches the raw secret directly. The 17-byte pad string “Key Pad for IKEv2” exists, the RFC explains, so that an implementation can store this derived value instead of the plaintext secret. Both sides compute this twice, once for InitiatorSignedOctets and once for ResponderSignedOctets, and each checks the other’s AUTH value against its own.

Because this whole exchange happens after the Diffie-Hellman handshake and inside an encrypted payload, an eavesdropper captures no AUTH value and no identity in the clear. This is the design flaw that doomed IKEv1’s aggressive mode, where the equivalent hash traveled unencrypted and let an attacker who captured one exchange crack the pre-shared key offline.

The AES-GCM Encrypted Payload

RFC 5282 updates RFC 7296’s Encrypted payload for authenticated-encryption algorithms. The structure is simpler than the original CBC-plus-separate-MAC design: a generic payload header, an 8-byte IV, and a ciphertext that already carries its own authentication tag:

Next Payload | C | Reserved | Payload Length
Initialization Vector (8 bytes)
Ciphertext (variable, includes the AES-GCM tag)

The AES-GCM nonce is the 4-byte salt from SK_ei or SK_er, concatenated with this 8-byte IV. The associated data is everything from the start of the IKE header through the four bytes of the Encrypted payload’s own generic header. This means the entire unencrypted part of the message, including the exchange type, the flags, and the message ID, is authenticated even though it is never encrypted. My code builds this associated data exactly, and seals the IDi and AUTH payloads of the IKE_AUTH exchange inside it. The responder’s independent recomputation of the AUTH value is what actually authenticates the initiator, not the AES-GCM tag alone, since the tag only proves the message was not tampered with in transit.

The Encapsulating Security Payload

Once the Child SA exists, data travels in ESP packets (RFC 4303), one UDP-free IP protocol of its own:

Security Parameters Index (4 bytes)
Sequence Number (4 bytes)
Payload Data (an 8-byte IV, then the ciphertext)
Padding, Pad Length, Next Header (the ESP trailer, encrypted)
Integrity Check Value (the AES-GCM tag)

RFC 4106 defines AES-GCM for ESP with the same salt-plus-IV nonce construction as RFC 5282’s IKE payload, and the same KEYMAT convention: a key with 4 extra salt bytes appended. The associated data this time is only the SPI and the sequence number, both sent in the clear but both authenticated. “Integrity Check Value” is the older name from the separate-MAC design. With AES-GCM, it is simply the 16-byte tag that the encryption itself produces. The padding rule in RFC 4303 section 2.4 is purely about alignment for a combined-mode cipher like AES-GCM. It adds enough bytes, numbered 1, 2, 3, and so on, to bring the Pad Length and Next Header fields to the end of a 4-byte word, so the authentication tag that follows starts on a 4-byte boundary.

Interactive Visualizer

The visualizer above runs one full exchange: the two IKE_SA_INIT messages, the key derivation, the two encrypted IKE_AUTH messages with their AUTH values, the Child SA keying material, and one encrypted ESP packet.

The fields are a seed for the keys, the two sides’ identities, the message to send, and a damage option.

• No damage. The exchange completes, and the final line shows the message the responder read back.
• Bad AUTH value. The initiator’s AUTH value is flipped by one bit before it is sent. The responder’s recomputed AUTH value no longer matches, and the exchange stops there.
• Tampered Diffie-Hellman value. The responder’s public Diffie-Hellman value is flipped by one bit after it was used to compute SKEYSEED, but before it is sent. The initiator then derives different SK_* keys than the responder actually used, so it cannot even open the encrypted IKE_AUTH response.
• Tampered ESP packet. The last byte of the encrypted ESP packet is flipped. The AES-GCM tag no longer matches, and the packet is rejected.
• The wrong direction’s ESP keys. The packet is sealed with the responder-to-initiator keys instead of the initiator-to-responder keys. The responder, which expects the other pair, rejects it.

The default seed and identities match the worked example below, so the log’s first run reproduces the same bytes this page prints. The page has its own SHA-256, HMAC, AES-256-GCM, and the 2048-bit Diffie-Hellman group.

A Worked Example

With the seed page-default and the identities initiator.example.com and responder.example.com, the exchange produces these SPIs and Diffie-Hellman public values.

Value	Bytes
Initiator’s SPI	7b6e20bfcf0e1cd8
Responder’s SPI	abcd3c1eefdea844
Initiator’s Diffie-Hellman public value, start	eb0ae4522ef2ab967ab0e48de5120c84
Responder’s Diffie-Hellman public value, start	cdba9134877001ab8c67718f44869446

Both IKE_SA_INIT messages are 368 bytes, including an SA payload of 40 bytes. The shared secret g^ir starts with ecf943f3fa383a7b5287df67e4518a69. From the nonces, the SPIs, and g^ir, both sides compute:

SKEYSEED = 7be424d797d6a93adc86b2ce726368f41ec5751fb01c07f77f16f45422b71f2b
SK_d      = 0c143719a3c0954b1ad7dfb7862fe411ceb62f5c7236208382b9528041a7943a

The encryption keys start with 51a5cc24b627f1d4f7f0c359fb219807 (initiator to responder) and 516551438ceb487decad49773fbe238a (responder to initiator). The initiator’s AUTH value is eca21c2de7806f82cb7ce56e9e939a7383e2c45a3fecc79fbd7a316b3b3612bc. The Child SA keying material starts with b67f296ba1edb575b00f62516ddfa18d, and the initiator-to-responder ESP key starts with b67f296ba1edb575b00f62516ddfa18d. The ESP packet carrying the message “hello over the tunnel” has the SPI c5ecb9e7, is 56 bytes long, and starts with c5ecb9e700000001fea24a8caffd638e.

A real NIST test vector

NIST’s own known-answer test file gives this case, with Ni = ed80dc79912c32a9, Nr = 35fb6d1a3feac078, and a g^ir starting with 42968e5d0ccc3cfc5a3e4bc1bba370ce, under SHA-256. The published SKEYSEED is:

37fde90b81d63692620f67367b62092e3efdc6a03509137d731004767ff35010

My code reproduces this exactly from the published inputs, and the 132-byte prf+ expansion of it, DKM, which starts with 63d018a08f7a29daa7d1f92cb63f45fa, also matches the file byte for byte.

Python Implementation

The first file is the key derivation of sections 2.13, 2.14, 2.15, 2.17, and 2.18: prf, prf+, SKEYSEED, the seven SK_* keys, Child SA keying material, the IKE SA rekey formula, and the pre-shared-key AUTH value.

# ikev2_kdf.py
# IKEv2 key derivation (RFC 7296 sections 2.13, 2.14, 2.15, 2.17, 2.18): the prf and prf+ constructions,
# SKEYSEED and the seven SK_* keys, Child SA keying material, IKE SA rekey, and the pre-shared-key AUTH value.
import hashlib
import hmac

HASHES = {"SHA-1": hashlib.sha1, "SHA-224": hashlib.sha224, "SHA-256": hashlib.sha256,
          "SHA-384": hashlib.sha384, "SHA-512": hashlib.sha512}


def prf_len(hash_name):
    return HASHES[hash_name]().digest_size


def prf(key, data, hash_name):
    return hmac.new(key, data, HASHES[hash_name]).digest()


def prf_plus(key, seed, length, hash_name):
    # the counter is a single octet, so prf+ is not defined beyond 255 blocks (RFC 7296 section 2.13)
    out, t = b"", b""
    counter = 1
    while len(out) < length:
        t = prf(key, t + seed + bytes([counter]), hash_name)
        out += t
        counter += 1
    return out[:length]


def skeyseed(ni, nr, g_ir, hash_name):
    return prf(ni + nr, g_ir, hash_name)


def derive_sk(skeyseed_value, ni, nr, spi_i, spi_r, hash_name, length):
    return prf_plus(skeyseed_value, ni + nr + spi_i + spi_r, length, hash_name)


def split_sk(dkm, hash_name, enc_key_len, integ_key_len):
    # RFC 7296 section 2.14: SK_d | SK_ai | SK_ar | SK_ei | SK_er | SK_pi | SK_pr, taken in order
    # for AES-GCM (RFC 5282), pass integ_key_len=0 and enc_key_len = AES key length + 4 salt bytes
    p = prf_len(hash_name)
    pos = 0
    out = {}
    for name, length in [("SK_d", p), ("SK_ai", integ_key_len), ("SK_ar", integ_key_len),
                          ("SK_ei", enc_key_len), ("SK_er", enc_key_len), ("SK_pi", p), ("SK_pr", p)]:
        out[name] = dkm[pos:pos + length]
        pos += length
    return out


def keymat_child(sk_d, ni, nr, hash_name, length, g_ir_new=None):
    seed = (g_ir_new + ni + nr) if g_ir_new is not None else (ni + nr)
    return prf_plus(sk_d, seed, length, hash_name)


def skeyseed_rekey(sk_d_old, g_ir_new, ni, nr, hash_name):
    return prf(sk_d_old, g_ir_new + ni + nr, hash_name)


KEY_PAD = b"Key Pad for IKEv2"


def auth_value(shared_secret, signed_octets, hash_name):
    # RFC 7296 section 2.15, pre-shared-key method
    padded = prf(shared_secret, KEY_PAD, hash_name)
    return prf(padded, signed_octets, hash_name)

The second file is the IKE message layer: the header, the generic payload header, the Security Association, Key Exchange, Nonce, Identification, and Authentication payloads, and the AES-GCM Encrypted payload of RFC 5282. The AES-GCM primitive itself, aes_gcm.py, is the same module the AES-GCM guide verified against NIST’s known-answer tests.

# ikev2_message.py
# IKEv2 wire formats (RFC 7296 section 3): the fixed header, the generic payload header, the Security
# Association payload (one proposal, AES-GCM so no separate integrity transform), Key Exchange, Nonce,
# Identification, Authentication, and the AES-GCM Encrypted payload of RFC 5282.
import aes_gcm

# Exchange types (section 3.1)
IKE_SA_INIT, IKE_AUTH, CREATE_CHILD_SA, INFORMATIONAL = 34, 35, 36, 37

# Payload type numbers (section 3.2)
PAYLOAD_NONE, PAYLOAD_SA, PAYLOAD_KE, PAYLOAD_IDi, PAYLOAD_IDr = 0, 33, 34, 35, 36
PAYLOAD_AUTH, PAYLOAD_Ni_Nr, PAYLOAD_SK = 39, 40, 46

# Transform types (section 3.3.2) and the IDs this implementation uses (IANA IKEv2 parameters registry)
TRANS_ENCR, TRANS_PRF, TRANS_DH = 1, 2, 4
ENCR_AES_GCM_16 = 20
PRF_HMAC_SHA2_256 = 5
DH_GROUP_14 = 14
ATTR_KEY_LENGTH = 14

# Identification types (section 3.5)
ID_FQDN, ID_RFC822_ADDR = 2, 3

# Authentication methods (section 3.8)
AUTH_SHARED_KEY_MIC = 2

FLAG_I, FLAG_R = 0x08, 0x20


def u32(n):
    return n.to_bytes(4, "big")


def u16(n):
    return n.to_bytes(2, "big")


def ike_header(spi_i, spi_r, next_payload, exchange_type, flags, message_id, length):
    return spi_i + spi_r + bytes([next_payload, 0x20, exchange_type, flags]) + u32(message_id) + u32(length)


def parse_ike_header(data):
    spi_i, spi_r = data[0:8], data[8:16]
    next_payload, ver, exchange_type, flags = data[16], data[17], data[18], data[19]
    message_id = int.from_bytes(data[20:24], "big")
    length = int.from_bytes(data[24:28], "big")
    return {"spi_i": spi_i, "spi_r": spi_r, "next_payload": next_payload, "exchange_type": exchange_type,
            "flags": flags, "message_id": message_id, "length": length}


def payload_header(next_payload, body_len):
    return bytes([next_payload, 0]) + u16(4 + body_len)


def parse_payload_header(data, pos):
    next_payload = data[pos]
    length = int.from_bytes(data[pos + 2:pos + 4], "big")
    return next_payload, length


def build_sa_payload(next_payload, key_length_bits=256):
    # one proposal: IKE, SPI size 0 (the SPI is the one in the IKE header), three transforms
    key_len_attr = bytes([0x80, ATTR_KEY_LENGTH]) + u16(key_length_bits)
    t_encr = bytes([3, 0]) + u16(8 + len(key_len_attr)) + bytes([TRANS_ENCR, 0]) + u16(ENCR_AES_GCM_16) + key_len_attr
    t_prf = bytes([3, 0]) + u16(8) + bytes([TRANS_PRF, 0]) + u16(PRF_HMAC_SHA2_256)
    t_dh = bytes([0, 0]) + u16(8) + bytes([TRANS_DH, 0]) + u16(DH_GROUP_14)
    transforms = t_encr + t_prf + t_dh
    proposal = bytes([0, 0]) + u16(8 + len(transforms)) + bytes([1, 1, 0, 3]) + transforms
    body = proposal
    return payload_header(next_payload, len(body)) + body


def build_ke_payload(next_payload, dh_group, public_value):
    body = u16(dh_group) + bytes(2) + public_value
    return payload_header(next_payload, len(body)) + body


def parse_ke_payload(data, pos, length):
    dh_group = int.from_bytes(data[pos + 4:pos + 6], "big")
    return dh_group, data[pos + 8:pos + length]


def build_nonce_payload(next_payload, nonce):
    return payload_header(next_payload, len(nonce)) + nonce


def build_id_payload(next_payload, id_type, id_data):
    body = bytes([id_type]) + bytes(3) + id_data
    return payload_header(next_payload, len(body)) + body


def id_payload_signed_part(id_type, id_data):
    # RestOfInitIDPayload / RestOfRespIDPayload: ID Type | RESERVED | ID data, without the generic payload header
    return bytes([id_type]) + bytes(3) + id_data


def build_auth_payload(next_payload, auth_method, auth_data):
    body = bytes([auth_method]) + bytes(3) + auth_data
    return payload_header(next_payload, len(body)) + body


def encrypted_payload_aad(ike_header_bytes, unencrypted_payloads, sk_header):
    return ike_header_bytes + unencrypted_payloads + sk_header


def seal_sk_payload(key, salt, iv, plaintext, aad, next_payload_of_first_inner):
    pad_len = 0   # RFC 5282: no alignment requirement, so no padding is needed
    body = plaintext + bytes([pad_len])
    nonce = salt + iv
    ciphertext = aes_gcm.seal(key, nonce, aad, body)
    sk_body = iv + ciphertext
    sk_header = bytes([next_payload_of_first_inner, 0]) + u16(4 + len(sk_body))
    return sk_header + sk_body


def open_sk_payload(key, salt, data, aad):
    iv = data[:8]
    nonce = salt + iv
    plaintext = aes_gcm.open_(key, nonce, aad, data[8:])
    pad_len = plaintext[-1]
    return plaintext[:len(plaintext) - 1 - pad_len]

The third file is ESP with AES-GCM (RFC 4303, RFC 4106).

# esp.py
# The Encapsulating Security Payload (RFC 4303) with AES-GCM (RFC 4106): the SPI/sequence-number header,
# the 8-byte explicit IV, AES-GCM encryption keyed by KEYMAT split into an AES key and a 4-byte salt, the
# padding and trailer, and the AAD covering the SPI and sequence number. AES-GCM itself is aes_gcm.py,
# the same module the AES-GCM article verified against NIST's known-answer tests.
import aes_gcm

NEXT_HEADER_IP4 = 4  # IP-in-IP, used in tunnel mode for this demonstration


def split_keymat_gcm(keymat):
    # RFC 4106 section 8.1: the AES-GCM-ESP key is the first bytes, the last 4 bytes are the salt
    return keymat[:-4], keymat[-4:]


def pack_header(spi, seq):
    return spi.to_bytes(4, "big") + seq.to_bytes(4, "big")


def seal(key, salt, spi, seq, iv, plaintext, next_header=NEXT_HEADER_IP4):
    pad_len = (-(len(plaintext) + 2)) % 4   # RFC 4303 section 2.4: pad so the trailer ends on a 4-byte boundary
    padding = bytes(range(1, pad_len + 1))  # the default 1, 2, 3, ... padding content of section 2.4
    body = plaintext + padding + bytes([pad_len, next_header])
    header = pack_header(spi, seq)
    nonce = salt + iv
    aad = header
    ciphertext_and_tag = aes_gcm.seal(key, nonce, aad, body)
    return header + iv + ciphertext_and_tag


def open_(key, salt, packet):
    spi, seq = int.from_bytes(packet[:4], "big"), int.from_bytes(packet[4:8], "big")
    iv = packet[8:16]
    nonce = salt + iv
    aad = packet[:8]
    plaintext = aes_gcm.open_(key, nonce, aad, packet[16:])
    pad_len, next_header = plaintext[-2], plaintext[-1]
    payload = plaintext[:len(plaintext) - 2 - pad_len]
    return spi, seq, payload, next_header

The key-derivation test runs every check that follows.

280 NIST CAVS IKEv2 KDF vectors match: SKEYSEED, DKM, DKM(Child SA), DKM(Child SA D-H), and SKEYSEED(Rekey), across SHA-1, SHA-224, SHA-256, SHA-384, and SHA-512
prf() also matches RFC 4231's own HMAC-SHA-256 and HMAC-SHA-512 test vectors 1 and 2
prf+: 50 random (key, seed) pairs confirm T1 equals prf(key, seed | 0x01), and a 300-byte expansion starts with the same T1
split_sk: the seven SK_* keys are all distinct and have the lengths the protocol requires
AUTH value (pre-shared key method): a changed shared secret always changes the AUTH value
IKEV2 KDF TESTS PASSED

I checked the code in layers.

• Key derivation against NIST. My implementation reproduces all 280 of NIST’s own CAVS known-answer test vectors for the IKEv2 key-derivation function, across SHA-1, SHA-224, SHA-256, SHA-384, and SHA-512. Each one checks SKEYSEED, the full prf+ expansion (DKM), the Child SA keying material with and without a fresh Diffie-Hellman exchange, and the IKE SA rekey formula.
• The PRF building block. prf() is plain HMAC, and it also matches RFC 4231’s own HMAC-SHA-256 and HMAC-SHA-512 test vectors.
• The Diffie-Hellman prime. The 2048-bit modulus matches RFC 3526 section 3 character for character, and it passes a Fermat primality check.
• ESP against the already-verified AES-GCM primitive. With zero padding, esp.seal reproduces a direct call to the AES-GCM module with the exact nonce and associated data that RFC 4106 specifies. The packet round-trips for a range of payload lengths, each ending on a 4-byte boundary, and tampering of either the ciphertext or the SPI is rejected.
• The message layer. The IKE header and every payload round-trip. The AES-GCM Encrypted payload round-trips its inner payloads, and rejects a changed ciphertext byte or a changed byte anywhere in the IKE header, since the header is part of the associated data even though it is sent unencrypted.
• A full exchange. A simulated IKE_SA_INIT and IKE_AUTH exchange, with a real Diffie-Hellman key agreement, produces matching SKEYSEED and SK_* values on both sides. It also produces a verified AUTH value in each direction, Child SA keying material, and one ESP packet that decrypts to the original message.

The page code has its own SHA-256, HMAC-SHA-256, AES-256-GCM, and the 2048-bit Diffie-Hellman group. Its SHA-256 and AES-256-GCM match Node’s crypto module on random cases, its HMAC-SHA-256 matches the same RFC 4231 vectors, its Diffie-Hellman prime matches the same RFC 3526 text, and its prf+ reproduces one of the real NIST CAVS vectors directly. Across all 7 CIPHER-level scenarios, including a bad AUTH value, a tampered Diffie-Hellman value, a tampered ESP packet, and the wrong direction’s ESP keys, the page’s log matches the Python reference message for message. All 7 of them agree on the final outcome too. I ran the page in a simulated browser.

What I did not check against an official source is the handshake as a whole, since neither RFC 7296 nor any NIST publication gives end-to-end vectors for a full IKE_SA_INIT and IKE_AUTH exchange. That part rests on two independent implementations, Python and JavaScript, agreeing with each other on every byte, and on the RFC’s own formulas, checked piece by piece. I did not run an interoperability test against a real IKEv2 implementation such as strongSwan or Libreswan.

Where It Stands

• The standard way to build a site-to-site VPN. IPsec remains the default choice where IKEv2 negotiation, strong authentication, and standardized key management matter, often alongside or instead of the simpler design of WireGuard.
• Layered like SSH, differently. The SSH guide covers another protocol with a negotiated key exchange and an authenticated-encryption data channel, though SSH carries a single TCP stream while ESP wraps IP packets directly.
• Built on the same primitives as elsewhere on this site. AES-GCM is also the subject of the dedicated AES-GCM guide, and classic Diffie-Hellman is the subject of the DH guide.

Limitations

This is a teaching implementation of the core exchange, not a production IPsec stack:

• Not constant time. The Python code uses variable-size integers for the modular exponentiation of Diffie-Hellman.
• One cipher suite. AES-256-GCM, PRF-HMAC-SHA2-256, and Diffie-Hellman group 14 throughout. No CBC ciphers, no separate integrity algorithms, no other Diffie-Hellman groups or elliptic-curve groups, and no certificate-based authentication.
• No negotiation logic. The code builds one SA payload offer and assumes it is accepted as sent. The real tie-breaking rules for multiple proposals are not implemented.
• No connection protocol beyond one Child SA. No rekeying, no additional CREATE_CHILD_SA exchanges, no Authentication Header, no tunnel-mode IP encapsulation beyond a placeholder next-header byte.
• No sockets. A real deployment carries IKE over UDP port 500 or 4500. This code builds and reads messages in memory and never opens a socket.
• No NAT traversal or cookies. The NAT detection and UDP encapsulation of RFC 7296 section 2.23 are missing. So is the COOKIE exchange of section 2.6, which lets a responder under a flood of requests avoid storing state.

Security Status

• IKEv1 aggressive mode, retired. The hash-disclosure weakness in pre-shared-key aggressive mode, which let an attacker who captured one exchange crack the key offline, was one reason the IETF moved IKEv1 to Historic status with RFC 9395 in 2023.
• The 2018 extension to main mode. Research presented at USENIX Security 2018, “The Dangers of Key Reuse”, showed that IKEv1’s supposedly safer main mode could also be broken offline. The attack needed a low-entropy pre-shared key and an active attacker, and is tracked as CVE-2018-5389.
• Logjam and weak Diffie-Hellman groups, 2015. Researchers found that millions of servers, including IKE VPN gateways, reused the same small set of 1024-bit and smaller Diffie-Hellman primes, making precomputation attacks practical. Their measurement estimated that about two-thirds of IPv4 IKEv1 hosts used a group vulnerable to this precomputation. This guide uses the 2048-bit group 14 for exactly that reason. RFC 8247 still makes group 14 mandatory to implement for IKEv2, and recommends group 19, a 256-bit elliptic-curve group, alongside it.
• IKEv2’s design avoids the aggressive-mode flaw. The AUTH payload only ever travels inside the AES-GCM Encrypted payload, after the Diffie-Hellman exchange has already produced session keys. So no identity or authentication data is ever visible to a passive eavesdropper, which rules out the IKEv1 aggressive-mode attack by construction.
• Host authentication is still the deployer’s job. A pre-shared key needs enough entropy to resist the kind of offline attack described above. RFC 7296 section 2.15 itself warns that a password-derived shared secret is typically insecure for exactly this reason.

FAQ

Why does ESP need a separate key from the IKE SA?

IKE keys protect only the negotiation itself, the IKE_SA_INIT and IKE_AUTH messages and anything else sent over the control channel. Every Child SA gets its own keys, derived from SK_d and fresh nonces, so compromising one Child SA’s keys does not expose the IKE SA or any other Child SA.

Could an eavesdropper read a session recorded today, later?

Not from the key exchange alone. The Diffie-Hellman exponents are ephemeral and are never transmitted or stored, so a later compromise of the long-term pre-shared key does not reveal a previously recorded session’s content, only that a MAC could now be forged for future traffic.

What is the difference between ESP and AH?

ESP, which this guide implements, provides confidentiality and integrity. The Authentication Header, AH, provides only integrity, with no encryption, and is rarely used today since most deployments that need authentication also want confidentiality.

Why does the Encrypted payload authenticate the whole IKE header, even the unencrypted part?

RFC 5282 section 5 makes the IKE header and every payload sent before the Encrypted payload part of the associated data. This means an attacker who flips a bit in the exchange type, the flags, or the message ID, none of which are encrypted, is still caught, because the AES-GCM tag would no longer verify.

What happens when the keys are renewed?

Both kinds of SA are rekeyed with a CREATE_CHILD_SA exchange. A new Child SA takes KEYMAT = prf+(SK_d, g^ir_new | Ni | Nr) from the existing IKE SA, with the new Diffie-Hellman value being optional. A new IKE SA computes SKEYSEED = prf(SK_d_old, g^ir_new | Ni | Nr) and then derives fresh SK_* keys as before. My code tests both formulas against NIST’s vectors but does not run the exchange itself.

Does a tampered packet reveal which byte was changed?

No, AES-GCM only reports pass or fail. My code’s damage modes each change a different field to show different failure points in the log, but a real implementation gives an attacker no information beyond “the tag did not match”.

References

1. RFC 7296, “Internet Key Exchange Protocol Version 2 (IKEv2),” by Kaufman, Hoffman, Nir, Eronen, and Kivinen, October 2014. Available at: https://www.rfc-editor.org/rfc/rfc7296

2. RFC 4303, “IP Encapsulating Security Payload (ESP),” by Kent, December 2005. Available at: https://www.rfc-editor.org/rfc/rfc4303

3. RFC 4106, “The Use of Galois/Counter Mode (GCM) in IPsec Encapsulating Security Payload (ESP),” by Viega and McGrew, June 2005. Available at: https://www.rfc-editor.org/rfc/rfc4106

4. RFC 5282, “Using Authenticated Encryption Algorithms with the Encrypted Payload of the Internet Key Exchange Protocol version 2 (IKEv2),” by Black and McGrew, August 2008. Available at: https://www.rfc-editor.org/rfc/rfc5282

5. RFC 3526, “More Modular Exponential (MODP) Diffie-Hellman Groups for Internet Key Exchange (IKE),” by Kivinen and Kojo, May 2003. Available at: https://www.rfc-editor.org/rfc/rfc3526

6. RFC 9395, “Internet Key Exchange Protocol Version 1 (IKEv1) and IPsec Documents to Historic Status,” by Migault, Guggemos, and Bormann, March 2023. Available at: https://www.rfc-editor.org/rfc/rfc9395

7. NIST Cryptographic Algorithm Validation Program, “IKEv2 KDF Test Vectors” (SP 800-135). Available at: https://csrc.nist.gov/projects/cryptographic-algorithm-validation-program/component-testing

8. RFC 4231, “Identifiers and Test Vectors for HMAC-SHA-224, HMAC-SHA-256, HMAC-SHA-384, and HMAC-SHA-512,” by Nystrom, December 2005. Available at: https://www.rfc-editor.org/rfc/rfc4231

9. Adrian, Bhargavan, Durumeric, et al. “Imperfect Forward Secrecy: How Diffie-Hellman Fails in Practice,” ACM CCS 2015 (the Logjam attack). Available at: https://weakdh.org/

10. RFC 8247, “Algorithm Implementation Requirements and Usage Guidance for the Internet Key Exchange Protocol Version 2 (IKEv2),” by Nir, Kivinen, Wouters, and Migault, September 2017. Available at: https://www.rfc-editor.org/rfc/rfc8247

11. Felsch, Grothe, Schwenk, Czubak, and Szymanek. “The Dangers of Key Reuse: Practical Attacks on IPsec IKE,” USENIX Security 2018. Available at: https://www.usenix.org/system/files/conference/usenixsecurity18/sec18-felsch.pdf