I’ve just added a new permanent page on the mysterious Blitz Ciphers to Cipher Mysteries. Basically, I discovered a few days ago that I had much higher resolution versions of the three scans so far released than I remembered having (i.e. 4MP rather than 1MP), which gave me a good-sized shove to put a proper page up for them. Also, a big hat tip to Edward B for asking if I had anything so useful as decent-sized scans. 😉
But that has also prompted me to revisit the issue of why the Blitz Ciphers aren’t apparently trivially crackable with normal crypto tools (e.g. zkdecrypto etc). And that prompted me to think again about how to go about detecting / predicting nulls.
As always, I had a look for null detection algorithms on the Internet, just in case there was some kind of magical analytical framework out there that I had somehow missed (there’s nothing in Cryptool etc). The closest mention I found was a 2005 message from Brian Tawney on the Voynich mailing list that basically described his independent version of my own homebrewed null character detector: which basically comes down to comparing the distribution of letters preceding / following nulls and seeing how closely that distribution approximates the context-less distribution of letters in the message.
For what it’s worth, my implementation of this hack predicts that B / D / E / M / S are the glyphs in the Blitz Ciphers most likely to be nulls: while another null-detector hack I wrote (that calculates the per-glyph difference in 1st order entropy if individual characters are removed from the stream) predicts that E / B / M / C / S / D may well be nulls.
So far so reasonable, you might initially think. But from my perspective, the problem with this is that nulls also behave a lot like vowels, in that they sit comfortably next to many different other letters / glyphs, can occur quite often within a ciphertext, and tend not to contain much information (in the Shannon entropy sense of the word). So I’m very far from convinced that I could tell nulls from vowels, or even from high-frequency nomenclature tokens (such as “the”, “and”, or perhaps “Freemason” 😉 ).
I might be wrong, but arguably the biggest theoretical problem with both of these hacks is that I think we would need to feed them a substantially larger ciphertext (I’m guessing 10x larger?) to get properly significant results in the presence of other cipher mechanisms (e.g. homophonic equivalents). Whereas a decent simple substitution cipher cracker can have a decent go at breaking a monoalphabetic cipher with as little as, say, 30 characters… so it may be clever, but it seems an order of magnitude cruder than proper cryptanalytical kit.
So… where are all the null detection algorithms? It seems to me that the cryptanalytical tools written these days are more focused on the statistical nuances of computer-era cryptography, while old school trickery (such as nulls and homophones) gets relatively little attention outside of the Zodiac Killer Ciphers world. Maybe there just aren’t any out there.
…unless you know better? 🙂