Breaking the Rail Fence Cipher

Breaking the Rail Fence Cipher: A Key Space You Can Just Exhaust

Introduction

The Rail Fence cipher’s own guide is blunt about its weaknesses: “Limited Key Space” and “Easy Cryptanalysis” sit right there in its pros-and-cons table, alongside the observation that the ciphertext is nothing more than an anagram of the plaintext. This post takes that claim at face value and builds the attack it implies. It turns out to be the most decisive breaker on this site, for a reason worth spelling out precisely.

Every other transposition cipher published here, Columnar Transposition, hides a much bigger search space behind what looks like a small key. A 6-column columnar key has 720 possible reading orders; a 9-column one has 362,880. That breaker has to brute-force every column count and every ordering within it. Rail Fence has no equivalent trapdoor. Its entire key is the number of rails: one integer, bounded above by the message length. Each rail count produces exactly one candidate decryption, not a factorial pile of them. There’s nothing to search within a key. There’s only the key itself to try, one value at a time, all the way through.

Table of Contents

• Why the Rail Fence Key Space Is So Small
• The Attack: Brute Force Every Rail Count
• Scoring a Candidate Decryption
• A Worked Example
• Python Implementation
• Interactive Visualizer
• Rail Fence vs. Columnar Transposition: Same Cipher Family, Very Different Search
• Limitations of This Attack
• FAQ
• References

Why the Rail Fence Key Space Is So Small

Encrypting with Rail Fence means walking the plaintext in a zigzag across a fixed number of rails, then reading each rail off top to bottom. Decrypting means the reverse: given the ciphertext and a rail count, there’s exactly one way to rebuild the zigzag and read the plaintext back off it. No ambiguity, no ordering choice, nothing left to decide once the rail count is fixed.

That means the only thing an attacker doesn’t know is the rail count itself. It has to be at least 2, since a 1-rail “zigzag” doesn’t scramble anything at all. And it can’t usefully exceed the message length minus one, since a rail count that large just walks straight down and back up once, barely mixing anything. For a 329-character message, that’s at most 327 candidate keys to check. That’s smaller than the simplified Enigma breaker’s 17,576 rotor positions by two orders of magnitude. It’s nowhere near the factorial blowup Columnar Transposition or Playfair’s 25! key square face. This is a key space small enough to exhaust completely, every single time, with room to spare.

The Attack: Brute Force Every Rail Count

Because the key space is so small and so cheaply enumerable, there’s no need for anything more sophisticated than trying every value:

1. Clean the ciphertext: uppercase it, and strip anything that isn’t a letter or digit, matching how the encryption-side visualizer itself only ever zigzags letters.
2. For each rail count from 2 up to a sane limit (the message length minus one, capped for display purposes), rebuild the zigzag skeleton to work out how many characters land on each rail, split the ciphertext into those rail-sized chunks, and read them back off in zigzag order. That’s the exact inverse of encryption.
3. Score each candidate decryption against how English-like it looks.
4. Keep whichever rail count scored highest. That’s the key, full stop. There’s no second stage to refine it further, because a rail count doesn’t need one.

Scoring a Candidate Decryption

Rail Fence leaves no spaces in the ciphertext, exactly like Columnar Transposition, so whole-word matching against a dictionary doesn’t apply. This breaker instead reuses the Columnar breaker’s exact scoring approach: count occurrences of common English substrings (the, and, ing, tion, and about twenty others, all three-plus letters so short fragments like “a” or “to” don’t match everywhere as noise) inside each candidate. The correct rail count reliably produces a decryption riddled with these fragments; every wrong rail count produces a scrambled anagram with only a handful of coincidental hits.

A Worked Example

The visualizer’s default ciphertext is 329 characters long. The theoretical upper bound on rails is the message length minus one, 328 here, but the search stops at 25: a practical display/search cap (max_rails in the code below), not a cryptographic one, since no real key that large ever needs checking in practice. Trying every rail count from 2 through 25 and scoring each decryption:

Rail count 7 wins by a landslide: a score of 22 against a runner-up of just 4, an 18-point margin with nothing else in the field even close. Decrypting with 7 rails recovers:

“Rail fence breakers are brute-forced by trying every possible rail count from two up to a limit and scoring each candidate decryption for how much it looks like real English, because the key space is only as large as the message length itself. This is the smallest key space of any breaker on this entire site, so a single pass through every possible rail count finds the answer with certainty every time.”

Re-encrypting that recovered plaintext with 7 rails reproduces the original ciphertext exactly, confirming the key.

Python Implementation

The interactive visualizer above runs this exact attack in JavaScript, with the same rail-count sweep and the same substring scoring. Here’s the same attack in Python.

Key Features

• Full brute force, not a heuristic search: every candidate rail count is tried and scored; nothing is skipped, pruned, or approximated, because the entire key space is small enough to afford that.
• Shares its scoring function with the Columnar breaker: both ciphers hide spaces from the ciphertext, so both need substring scoring rather than whole-word matching.
• One candidate per key: the Columnar breaker has to try every column ordering for each column count. This breaker’s inner loop has nothing left to search: decryption is fully determined the moment the rail count is chosen.

Code

# railfence_breaker.py
#
# Breaks a Rail Fence cipher by brute force. The entire key is the number
# of rails, and it's bounded by the message length -- there's no larger
# search space hiding behind it the way Columnar Transposition hides a
# factorial number of column orderings behind its column count. So this
# breaker just decrypts with every rail count from 2 up to a sane limit
# and keeps whichever result scores highest against common English
# substrings (no spaces survive this cipher, so whole-word matching
# doesn't apply).

COMMON_SUBSTRINGS = ['the', 'and', 'ing', 'her', 'was', 'for', 'that', 'with',
                      'you', 'this', 'have', 'from', 'not', 'are', 'but', 'all',
                      'they', 'one', 'his', 'ent', 'tion', 'ere']


def clean(text):
    return ''.join(c for c in text.upper() if c.isalnum())


def build_fence(text, num_rails):
    fence = [[] for _ in range(num_rails)]
    rail, direction = 0, 1
    for ch in text:
        fence[rail].append(ch)
        rail += direction
        if rail == 0 or rail == num_rails - 1:
            direction *= -1
    return fence


def encrypt(text, num_rails):
    fence = build_fence(text, num_rails)
    return ''.join(ch for rail in fence for ch in rail)


def decrypt(ciphertext, num_rails):
    rail_lengths = [len(rail) for rail in build_fence(ciphertext, num_rails)]
    fence, idx = [], 0
    for length in rail_lengths:
        fence.append(list(ciphertext[idx:idx + length]))
        idx += length

    result, counters = [], [0] * num_rails
    rail, direction = 0, 1
    for _ in ciphertext:
        result.append(fence[rail][counters[rail]])
        counters[rail] += 1
        rail += direction
        if rail == 0 or rail == num_rails - 1:
            direction *= -1
    return ''.join(result)


def score(text):
    lower = text.lower()
    total = 0
    for word in COMMON_SUBSTRINGS:
        idx = 0
        while True:
            idx = lower.find(word, idx)
            if idx == -1:
                break
            total += 1
            idx += 1
    return total


def break_railfence(ciphertext, max_rails=25):
    upper = min(max_rails, len(ciphertext) - 1)
    best = {'rails': None, 'score': -1, 'plaintext': ''}
    results = []
    for rails in range(2, upper + 1):
        plaintext = decrypt(ciphertext, rails)
        s = score(plaintext)
        results.append((rails, s))
        if s > best['score']:
            best = {'rails': rails, 'score': s, 'plaintext': plaintext}
    return best, results


if __name__ == '__main__':
    ciphertext = "RAEIITARIIHRESSEITEBSOTPCEEYAEKTFYNSBNFOLOIDDTOCIEEBCYPALMSHTSHKYYRIESASHYOLOHACRRTIREUORGSLURTICNNAPNUTKAHAEAYAESTSIETSNEHNESSRRSIUTNHTEILBRRRTEOEOOPMSGATYFMLILSUKCLRHAGESSSPAATTTIAOESANSSTAVMFESBCYVPRCMUIDECEROWOLEISEENGTGNLIMEAFKNIINPUVIRTDWIIEEECAEEBEYALTOTNAHDCROOSNLEHIOESEEFHALCOEORSGEGEBEFNEWNYNRDRIWACEHKGTSALTLERELHLIRT"
    best, results = break_railfence(ciphertext)
    print('Best rail count:', best['rails'], 'score:', best['score'])
    runner_up = sorted([r for r in results if r[0] != best['rails']], key=lambda x: -x[1])[0]
    print('Runner-up:', runner_up)
    print()
    print('Recovered plaintext:')
    print(best['plaintext'])
    print()
    print('Round-trip check:', encrypt(best['plaintext'], best['rails']) == ciphertext)

Running this against the same demo ciphertext as the visualizer produces:

Best rail count: 7 score: 22
Runner-up: (3, 4)

Recovered plaintext:
RAILFENCEBREAKERSAREBRUTEFORCEDBYTRYINGEVERYPOSSIBLERAILCOUNTFROMTWOUPTOALIMITANDSCORINGEACHCANDIDATEDECRYPTIONFORHOWMUCHITLOOKSLIKEREALENGLISHBECAUSETHEKEYSPACEISONLYASLARGEASTHEMESSAGELENGTHITSELFTHISISTHESMALLESTKEYSPACEOFANYBREAKERONTHISENTIRESITESOASINGLEPASSTHROUGHEVERYPOSSIBLERAILCOUNTFINDSTHEANSWERWITHCERTAINTYEVERYTIME

Round-trip check: True

This matches the visualizer’s own result exactly.

For Fun: The Same Thing in About 20 Lines

Same spirit as this site’s other golfed bonus sections: not for learning the algorithm from, just a demonstration of how little code an exhaustive search over a genuinely small key space actually needs.

W = ['the','and','ing','her','was','for','that','with','you','this','have','from','not','are','but','all','they','one','his','ent','tion','ere']

def fence(t, n):
    f, r, d = [[] for _ in range(n)], 0, 1
    for c in t:
        f[r].append(c); r += d
        if r in (0, n-1): d *= -1
    return f

def dec(t, n):
    L = [len(x) for x in fence(t, n)]
    i = 0; F = []
    for l in L:
        F.append(list(t[i:i+l])); i += l
    o, C, r, d = [], [0]*n, 0, 1
    for _ in t:
        o.append(F[r][C[r]]); C[r] += 1; r += d
        if r in (0, n-1): d *= -1
    return ''.join(o)

def sc(t):
    t = t.lower()
    return sum(t.count(w) for w in W)

def crack(ct, mx=25):
    return max((dec(ct, n) for n in range(2, min(mx, len(ct)-1)+1)), key=sc)

if __name__ == '__main__':
    ciphertext = "RAEIITARIIHRESSEITEBSOTPCEEYAEKTFYNSBNFOLOIDDTOCIEEBCYPALMSHTSHKYYRIESASHYOLOHACRRTIREUORGSLURTICNNAPNUTKAHAEAYAESTSIETSNEHNESSRRSIUTNHTEILBRRRTEOEOOPMSGATYFMLILSUKCLRHAGESSSPAATTTIAOESANSSTAVMFESBCYVPRCMUIDECEROWOLEISEENGTGNLIMEAFKNIINPUVIRTDWIIEEECAEEBEYALTOTNAHDCROOSNLEHIOESEEFHALCOEORSGEGEBEFNEWNYNRDRIWACEHKGTSALTLERELHLIRT"
    print(crack(ciphertext))

crack collapses the whole sweep into a single max() call over a generator, using sc (which itself collapses to a one-liner via str.count) as the key function. Python does the “try every rail count, keep the highest-scoring one” bookkeeping that the readable version spells out explicitly in its break_railfence loop. Verified to produce the byte-for-byte identical recovered plaintext as the readable version and the visualizer.

Interactive Visualizer

Try it above: paste any Rail Fence ciphertext (or use the default), pick how high to let the rail count search go, and watch every candidate get decrypted and scored in a single pass. There are no restarts, no annealing, no ambiguity about whether the search converged. The bar chart makes the winning margin obvious at a glance.

Rail Fence vs. Columnar Transposition: Same Cipher Family, Very Different Search

Both ciphers are transpositions that leave letter frequencies untouched and hide spaces from the ciphertext. That’s exactly why they share a scoring function. What separates them is entirely about how much the key implies: Columnar’s key describes an ordering over several items, so the key space explodes combinatorially with the column count. Rail Fence’s key is just a magnitude, so its key space grows only linearly with the message length. That structural difference is the entire reason one breaker needs restarts-and-permutations machinery while the other is a single, short, always-terminating loop.

Limitations of This Attack

This breaker’s cleaning step strips anything that isn’t a letter or digit, matching the encryption-side visualizer’s own behavior. A ciphertext that was produced with a different alphabet or that mixes in punctuation the encryptor intended to preserve would need its own handling first.

Scoring depends on having enough ciphertext for common substrings to show up by chance at the right rail count and not spuriously at the wrong ones. Very short messages (under roughly 20-30 characters) don’t give the scorer much to work with. A handful of wrong rail counts might tie or edge out the correct one on raw luck, the same limitation the Columnar breaker has for the same underlying reason. In practice this is rarely an issue for Rail Fence specifically, since the key space is small enough that even a somewhat noisy scoring signal still tends to separate the true answer from the pack (see the 22-vs-4 margin in the worked example above).

Finally, this attack is specific to the basic Rail Fence pattern: a fixed zigzag with two fixed turning points at the top and bottom rail. Variants that start the zigzag partway down, skip rails, or otherwise modify the walking pattern would need the rail-rebuilding logic in build_fence/decrypt adjusted to match. The core idea, that the key space is small so you just try all of it, would still apply.

FAQ

Why doesn’t this breaker need simulated annealing or statistical hill-climbing like the Substitution or Playfair breakers?

Because it doesn’t need to search at all in the way those breakers do. Substitution’s 26! and Playfair’s 25! key spaces are far too large to check exhaustively, so those breakers hill-climb toward a good answer using statistics as a fitness signal. Rail Fence’s key space, at most a few hundred values for any realistic message, is small enough to check every single value directly, so there’s nothing to approximate.

How is this different from the Columnar Transposition breaker, if they’re both transposition ciphers?

Columnar’s key names a column count and an ordering of those columns, so its key space grows factorially (720 orderings for 6 columns, 362,880 for 9). Rail Fence’s key is just the rail count: one number, no ordering to choose, so its key space grows only linearly with the message length. Both breakers score candidates the same way (common substrings, since neither cipher preserves spaces), but Rail Fence’s outer loop has nothing left to search once a rail count is picked.

What’s the maximum number of rails worth trying?

The message length minus one. A rail count equal to or greater than that just walks straight down the message and back up once. That barely rearranges anything, and it never actually appears as the “correct” answer for any real key. So capping the search there loses nothing.

Can this breaker be fooled by a message that happens to score well at the wrong rail count?

In principle, yes: like any statistical scorer, an unlucky short ciphertext could produce a coincidental high score at the wrong rail count. In practice, this is far less likely here than for the site’s other breakers, because the number of candidates is so small that spurious ties are rare. The true rail count’s decryption tends to win by a wide margin (see the worked example’s 22-vs-4 split) rather than by a hair.

Does this attack need any known plaintext, the way the Hill cipher breaker does?

No. This is a ciphertext-only attack, in the same category as the Caesar, Substitution, Playfair, and simplified Enigma breakers on this site. It works purely by trying every possible key and scoring the result; unlike the Hill breaker, it never needs the attacker to already know part of the plaintext.

References

1. Wikipedia. “Rail fence cipher” (Cryptanalysis considerations). Available at: https://en.wikipedia.org/wiki/Rail_fence_cipher

2. Practical Cryptography. “Rail Fence Cipher.” Available at: http://practicalcryptography.com/ciphers/rail-fence-cipher/

3. dCode. “Rail Fence (Zig-Zag) Cipher - Online Decoder, Encoder, Solver.” Available at: https://www.dcode.fr/rail-fence-cipher