Earlier articles have mentioned checksums four times already — Ethernet has a check field at the end of the frame, IP has a checksum covering only its header, TCP and UDP have one covering the whole segment, and the layers article used TCP's checksum as proof that the layers leak.
None of them answered the question that should come first — what does it actually catch, and what does it miss?
The short answer is that the internet checksum is far weaker than people assume. There are kinds of corruption it cannot detect at all — not sometimes, but never, every single time — and that is not a bug. It follows from properties a 1978 design document lists as advantages.
This article proves it by running two hundred thousand trials.
If you have never looked, start here
The internet checksum can be computed in a few lines with no library at all.
$ python3 -c "
import struct
def ck(b):
if len(b)%2: b += b'\0'
s = sum(struct.unpack('!%dH' % (len(b)//2), b))
while s >> 16: s = (s & 0xffff) + (s >> 16)
return (~s) & 0xffff
print(hex(ck(b'HELLO WORLD!')))"That gives 0x2e91 — take the bytes two at a time, add them all, fold the carry back in, invert every bit. Done.
That simplicity is both its strength and its weakness.
Corruption comes in several shapes, not one
When people say data got corrupted they usually picture a single flipped bit. On a real wire, damage comes in several shapes, and each one challenges a detection scheme differently.
single-bit one bit flips
burst several adjacent bits fail together, from brief noise
reorder data arrives in a different order
duplicate the same piece arrives twice
truncate data is cut shortA method that is good against one shape can be useless against another, which is the whole point of this article.
Parity — the simplest, and the easiest to fool
The oldest method counts the one-bits and adds one more bit to make the total even. The receiver counts again; if it is not even, something broke.
Its limit follows straight from the definition — it only catches an odd number of flipped bits. Flip two and the total is even again, and parity reports everything is fine.
I ran two hundred thousand trials.
flip 1 bit caught every time
flip 2 bits never caught, not once
flip 3 bits caught every time
flip 4 bits never caught, not onceNot 99% or 1%, but 0% and 100% alternating with odd and even — perfectly predictable behaviour, and very bad if real damage tends to arrive in bursts.
The internet checksum
RFC 1071 of September 1988, Computing the Internet Checksum, describes the method and the reasoning behind it. It uses one's complement addition, which differs from ordinary addition in that the carry is folded back into the lowest bit.
The document explains why this was chosen over XOR.
"One's complement arithmetic is better than two's complement because it is equally sensitive to errors in all bit positions."
"It is just this property that makes some sort of addition preferable to a simple exclusive-OR which is frequently used but permits an even number of drops (pick ups) in any bit channel."
XOR has parity's problem: an even number of errors cancels itself out. Addition fixes that, at the cost of a new problem.
Four properties, of which the first is the weakness
The appendix of RFC 1071 is an older document, IEN 45, dated 5 June 1978 and written by William W. Plummer, listing four properties the addition operator must have. The first is:
"(P1) + is commutative. Thus, the order in which the 16-bit bytes are 'added' together is unimportant."
Read that again slowly. The order of the words does not matter — written as an advantage, because it lets an implementation walk memory in either direction and compute quickly on a variety of machines.
But the same sentence means something else too: if order does not matter to the computation, then reordering cannot be detected.
The others follow the same reasoning.
P2 an identity element, so the sender zeroes the field then computes
P3 an inverse, so the receiver can compute and expect zero
P4 associativity, so the checksum field can sit anywhereTogether the four make the checksum fast and flexible — and blind in several places at once.
The experiment: what is caught, what is not
I took one sentence and damaged it four ways, then measured whether each method noticed. ok means caught, MISS means missed.
corruption parity checksum CRC-32
flip 1 bit ok ok ok
flip 2 bits in one byte MISS ok ok
swap two 16-bit words MISS MISS ok
swap word 1 and word 5 MISS MISS okThe bottom two rows are P1 in action.
Think about what that means in practice. If the message is a payment instruction with the account number and the amount in different words, swapping those two words changes everything — and the checksum reports it as fine, every time.
The 3.1% that is not a coincidence
Two hundred thousand trials on random 64-byte messages, counting only the cases that went undetected.
corruption parity checksum CRC-32
flip 1 bit 0 ( 0.0%) 0 ( 0.0%) 0
flip 2 bits 200,000 (100.0%) 6,282 ( 3.1%) 0
flip 3 bits 0 ( 0.0%) 590 ( 0.3%) 0
flip 4 bits 200,000 (100.0%) 651 ( 0.3%) 0
swap two words 199,995 (100.0%) 199,995 (100.0%) 0The 3.1% on the two-bit row is the interesting one, because if damage were purely random a 16-bit checksum should miss about 1 in 65,536, or 0.0015%. What was measured is two thousand times worse than that.
I chased the mechanism, and it explains the number completely.
If the two flipped bits sit at the same position within their 16-bit words and flip in opposite directions, one adds 2^k and the other subtracts exactly 2^k. The total is unchanged, so the checksum is unchanged.
The odds work out directly.
1 in 16 same bit position
1 in 2 opposite directions
1 in 32 together, which is 3.1%And counting from the trial data: the number of cases meeting that condition was 6,117, and the number of checksum misses was 6,117 exactly. Not close — identical, case for case.
CRC — stronger, and living one layer down
Ethernet does not use an additive checksum. It uses CRC-32, computed as polynomial division in binary rather than addition, so it has no commutative property to exploit.
In the same trials, CRC-32 caught every case — one million runs without a single miss.
That the layers chose different methods is not inconsistency; it is a different trade.
Ethernet CRC-32 whole frame done in hardware, costs no CPU
IP 16-bit sum header only routers recompute at every hop
TCP 16-bit sum header+data done in software, must be fastWhy accept something weaker higher up
The answer lives in properties P2 and P4 above — because routers have to edit the header at every hop.
Every time a packet passes a router, TTL drops by one, which means the IP checksum must be recomputed. With CRC-32 the router would have to read the whole packet and start over each time; with this kind of addition it can adjust only the part that changed and never touch the other bytes.
That is a trade of detection strength against speed of editing in transit, decided back when a router was a PDP that did arithmetic slowly. IEN 45 even lists memory cycles per word in the document.
RFC 1071 states one more property that explains why it had to be this shape.
"(P6) Adding the checksum to a packet does not change the information bytes."
"This property allows intermediate computers such as gateway machines to act on fields (i.e., the Internet Destination Address) without having to first decode the packet."
A router can read the destination address immediately without decoding anything first — something certain error-correcting codes cannot offer.
Two zeros, and a bug that lived in an RFC for four years
One's complement arithmetic has an oddity: it has two zeros.
+0 0x0000
-0 0xFFFFBoth represent zero, and that difference produced a bug that sat in a standards document for four years.
RFC 1141 of 1990 offered a shortcut letting routers adjust the checksum without recomputing it entirely. The formula works in almost every case, except when the result is zero. RFC 1624 of May 1994 corrected it and explained why:
"RFC 1141 yields an updated header checksum of -0 when it should be +0. This is because it assumed that one's complement has a distributive property, which does not hold when the result is 0"
A formula that looks right is wrong only when the result is zero, and nobody noticed for four years because that case is rare. That is the signature of an arithmetic bug — it does not fail constantly. It waits.
When it lies
"The checksum passed, so the data is correct." No. It means the data was not damaged in a way this method can see, which is a different statement.
"A 16-bit checksum misses 1 in 65,536." True only if damage is purely random. Real damage has structure, and the measured figure above is 3.1% in one case.
"Ethernet already checked it, so the upper layers need not." Ethernet's CRC covers one cable's worth of travel. At the router the frame is unwrapped and rebuilt with a fresh CRC, so corruption that happens inside the router's memory gets a new, perfectly valid CRC for the now-wrong data. This is exactly why an end-to-end checksum has to exist as well.
"CRC is a form of encryption." Not remotely. Anyone can recompute a CRC. It detects accidents, not intent. To stop someone deliberately editing data you need signatures, as the TLS article describes.
"UDP's checksum can be turned off, it's fine." On IPv4 it can. On IPv6 it cannot, because IPv6 removed the header checksum entirely — if the transport layer does not check, nothing does.
Real cases from real work
Case 1 — data is corrupted while everything reports healthy
Situation Files copied over the network are occasionally corrupt, but no error counter anywhere shows anything.
How to read it Switch counters only count frames whose CRC failed. If the damage happens inside an intermediate device's memory rather than on the wire, the frame gets its CRC computed after it was already wrong, so the counters stay clean — and TCP's checksum has the miss rate shown above. The only way to prove it is to compare a hash of the file at both ends.
What this does not prove Different hashes say the data is wrong; they do not say where it went wrong. It could be disk, memory, or network. Each has to be tested separately.
Case 2 — choosing a method for your own work
Situation You are designing a protocol for an embedded device.
How to read it Ask two questions before choosing.
1 what shape of damage do you expect
2 does anything need to edit the data in transitIf nothing edits in transit, choose a CRC: much stronger, and most hardware computes it for free. If an intermediate device must edit some field, a 16-bit sum may be worth it for the same reason IP chose one.
What this does not prove The numbers here come from corruption I generated randomly. Real damage on a wire tends to arrive in adjacent bursts, which is exactly what CRCs are designed for — and this article has not tested that case.
Case 3 — proving the blindness to somebody
Situation You need to explain to a team why another hash layer is needed.
Command Let them run it. Ten seconds.
$ python3 -c "
import struct
def ck(b):
s = sum(struct.unpack('!%dH' % (len(b)//2), b))
while s >> 16: s = (s & 0xffff) + (s >> 16)
return (~s) & 0xffff
a = b'AABB'
b = b'BBAA'
print(a, hex(ck(a)))
print(b, hex(ck(b)))"Real output
b'AABB' 0x7c7c
b'BBAA' 0x7c7cHow to read it Different data, identical checksum, because P1 said the order does not matter. Seeing it with your own eyes ends the argument faster than explaining it.
What this does not prove The example shows it is possible; it says nothing about how often it happens on a real network. Something has to cause words to be swapped, and that is far rarer than a flipped bit.
When detection is not enough
Detection tells you only that something broke; it neither locates nor repairs it. There are two ways forward — ask for a retransmission, which is what TCP does, or add enough redundancy that the receiver can repair it, which is what satellite links and ECC memory do.
The first is much cheaper when retransmission is fast. The second wins when retransmission is expensive or too slow. And neither helps at all if the other end is deliberately editing the data.
References
Standards
- RFC 1071, September 1988, Computing the Internet Checksum — the method, and an appendix reproducing IEN 45 of 1978 which lists properties P1 through P6 quoted here
- RFC 1141, January 1990, the shortcut for adjusting a checksum, with the flaw
- RFC 1624, May 1994, which fixed it, with the explanation about the two zeros
- RFC 9293, the requirement that TCP's checksum can never be turned off