HKDF: HMAC-Based Key Derivation

HKDF: HMAC-Based Key Derivation

Introduction

Real protocols almost never need just one key. TLS needs separate keys for encryption and authentication, in each direction. The Signal Protocol derives a fresh key for every single message via its Double Ratchet. Almost universally, these systems start with one shared secret (the output of a Diffie-Hellman exchange, for instance) and need to turn it into several independent, cryptographically strong keys. HKDF (HMAC-based Key Derivation Function), standardized in RFC 5869, is the standard tool for exactly this job.

Table of Contents

• The Extract-then-Expand Design
• Step 1: Extract
• Step 2: Expand
• Why Separate Extract and Expand?
• A Verified Example
• Python Implementation
• Limitations
• Real-World Applications
• FAQ
• References

The Extract-then-Expand Design

HKDF splits key derivation into two distinct steps, each doing one job well:

1. Extract: concentrate the entropy from a (possibly not perfectly random) input, like a raw Diffie-Hellman shared secret, into a fixed-size, uniformly random-looking pseudorandom key (PRK).
2. Expand: stretch that PRK into as many bytes of output key material as needed, optionally binding in context information to produce distinct keys for distinct purposes from the same PRK.

Both steps are built entirely from HMAC (covered in its own HMAC guide) with a chosen underlying hash function. There are no new cryptographic primitives, just a carefully specified way of combining an already-trusted one.

Step 1: Extract

PRK = HMAC-Hash(salt, input_key_material)

The salt is optional (an all-zero value is used if omitted) but strongly recommended when available. Even a non-secret salt meaningfully strengthens the extraction step, especially when the input key material’s randomness quality isn’t perfectly guaranteed.

Step 2: Expand

T(0) = empty string
T(1) = HMAC-Hash(PRK, T(0) | info | 0x01)
T(2) = HMAC-Hash(PRK, T(1) | info | 0x02)
T(3) = HMAC-Hash(PRK, T(2) | info | 0x03)
...
OKM = T(1) | T(2) | T(3) | ... truncated to the requested length

The info parameter lets you derive multiple independent keys from the same PRK just by varying a context string. For example, you can derive a distinct encryption key and a distinct authentication key from the same underlying shared secret, without needing two separate extraction steps.

Interactive Visualizer

The visualizer above runs the real HKDF-SHA256 construction: genuine extract and expand steps built on real HMAC-SHA256. It was verified against a standard cryptographic library’s HKDF implementation before being published here.

Why Separate Extract and Expand?

Input key material isn’t always uniformly random. A Diffie-Hellman shared secret, for instance, is a specific number within a mathematically structured group, not an arbitrary random string, even though it’s unpredictable to an attacker. The Extract step’s entire job is converting that structured, possibly-uneven input into something statistically indistinguishable from a uniformly random key before any further derivation happens. That’s a formally analyzed property, and it gives HKDF’s overall security proof a clean foundation to build on, rather than hoping the raw input’s randomness properties are good enough on their own.

A Verified Example

Using RFC 5869-style parameters:

• Input key material: “input key material”
• Salt: “salt value”
• Info: “context info”
• Output length: 42 bytes

produces a specific, deterministic 42-byte output key material value. That’s exactly what the visualizer above reproduces, matching a standard cryptographic library’s HKDF-SHA256 output byte for byte.

Python Implementation

The Extract and Expand formulas above translate directly into a working implementation, built on the same HMAC-SHA256 from the HMAC guide’s Python Implementation:

import hashlib

BLOCK_SIZE = 64

def hmac_sha256(key: bytes, message: bytes) -> bytes:
    if len(key) > BLOCK_SIZE:
        key = hashlib.sha256(key).digest()
    key = key + b'\x00' * (BLOCK_SIZE - len(key))
    ipad = bytes(b ^ 0x36 for b in key)
    opad = bytes(b ^ 0x5c for b in key)
    inner_hash = hashlib.sha256(ipad + message).digest()
    return hashlib.sha256(opad + inner_hash).digest()

def hkdf_extract(salt: bytes, ikm: bytes) -> bytes:
    return hmac_sha256(salt, ikm)

def hkdf_expand(prk: bytes, info: bytes, length: int) -> bytes:
    t = b''
    okm = b''
    counter = 1
    while len(okm) < length:
        t = hmac_sha256(prk, t + info + bytes([counter]))
        okm += t
        counter += 1
    return okm[:length]

def hkdf_sha256(ikm: bytes, salt: bytes, info: bytes, length: int) -> bytes:
    prk = hkdf_extract(salt, ikm)
    return hkdf_expand(prk, info, length)

if __name__ == "__main__":
    ikm = b"input key material"
    salt = b"salt value"
    info = b"context info"

    okm = hkdf_sha256(ikm, salt, info, length=42)
    print(f"OKM: {okm.hex()}")

Running this with the exact parameters from the Verified Example above produces de3847033eebae29b138a23dd47f8923a905f55fd92089451344713447e78bf5d20db7b172ffdb2e5f31. I checked this implementation against pycryptodome’s own HKDF function before writing this up, and the two agree exactly.

Limitations

This uses hashlib.sha256 for the underlying hash (see the SHA-2 guide’s Python Implementation for a from-scratch version of that), so what’s left is genuinely the extract/expand structure HKDF adds:

• Fixed to HKDF-SHA256. RFC 5869 defines HKDF generically over any hash function usable with HMAC; this code only implements the SHA-256 instantiation, the most common choice in modern protocols.
• No maximum-length check. RFC 5869 caps the Expand step’s output at 255 times the hash length (since the counter byte would otherwise overflow); this code will happily loop past that point and produce a bytes([counter]) that raises an error once counter exceeds 255, rather than failing with a clear message earlier.
• No streaming interface. This function takes the whole ikm as one bytes object; a real implementation might need to derive keys from data assembled incrementally.
• Real applications should use a vetted library’s HKDF, not this. This code exists to make the extract/expand split inspectable in Python, not to replace an audited implementation like cryptography’s or pycryptodome’s.

Real-World Applications

• TLS 1.3: uses HKDF extensively throughout its key schedule, deriving all traffic keys, handshake keys, and various intermediate secrets from the initial (EC)DHE shared secret.
• Signal Protocol: the Double Ratchet’s per-message key derivation is built on HKDF, expanding chain keys into message keys with each step.
• WireGuard: uses HKDF (via its Noise Protocol Framework foundation) for deriving session keys during its handshake.
• General protocol design: whenever a system has one shared secret and needs several independent-looking keys for different purposes, HKDF is close to the default, well-analyzed answer.

FAQ

What’s the difference between HKDF and PBKDF2?

They solve different problems: HKDF is designed to expand an already-high-entropy secret (like a Diffie-Hellman output) into multiple derived keys efficiently. PBKDF2 is designed to slow down brute-forcing a low-entropy password, deliberately taking much longer per derivation. Using HKDF directly on a user password (skipping the deliberate slowness) would be insecure; using PBKDF2 where HKDF’s speed and multi-key expansion are needed would be needlessly slow and awkward.

Is the salt required for HKDF?

No, but it’s recommended whenever one is available. RFC 5869 defines the behavior when no salt is provided (defaulting to a string of zero bytes), and even a non-secret, non-random salt still meaningfully improves the Extract step’s security properties.

What’s the “info” parameter actually for?

It’s a context-binding string that lets you derive multiple, cryptographically independent keys from a single PRK, for example “encryption key” versus “authentication key” as the info string. You don’t need to run the (potentially more expensive) Extract step more than once.

Can HKDF be used with any hash function?

Yes. RFC 5869 defines HKDF generically over any hash function usable with HMAC; HKDF-SHA256 is the most common choice in modern protocols, though HKDF-SHA1 and other variants exist in older or legacy systems.

Why not just use the shared secret directly as a key?

A raw Diffie-Hellman shared secret has structure (it’s an element of a specific mathematical group) that a truly random key wouldn’t have. Using it directly also provides no way to derive multiple distinct keys for multiple purposes from a single exchange. HKDF’s extract step normalizes that structure away, and the expand step handles multi-key derivation cleanly.

References

1. RFC 5869. “HMAC-based Extract-and-Expand Key Derivation Function (HKDF).” IETF, 2010. Available at: https://datatracker.ietf.org/doc/html/rfc5869

2. Krawczyk, H. “Cryptographic Extraction and Key Derivation: The HKDF Scheme.” CRYPTO 2010. This is the academic paper behind HKDF’s design and security analysis.

3. Wikipedia. “HKDF.” Available at: https://en.wikipedia.org/wiki/HKDF