All about the Chaocipher…

by ⋅ 8 Comments

I’m getting a bit cheesed off with the Internet: every time I do a search for anything Cipher Mysteries-ish, it seems that half Google’s hits are for ghastly sites listing “Top 10 Unsolved Mysteries” or “10 Most Bizarre Uncracked Codes“. Still, perhaps I should be more grateful to the GooglePlex that I’m not getting “Top 10 Paris Hilton Modesty Tips” and its tawdry ilk.

Realistically, there is only one uncracked code/cipher listing on the web from which all the rest get cut-and-pasted: Elonka’s list of famous unsolved codes and ciphers. But Elonka Dunin has long since moved on (coincidentally, she went from cryptography into computer game production at about the same time that I made the reverse journey), which is perhaps why all of these lists look a bit dated. Perhaps I should do my own list soon (maybe, if I had the time).

Happily, Elonka did manage to nail most of the usual suspects: the Beale Papers, the Voynich Manuscript, Dorabella, Zodiac Killer, d’Agapeyeff, Phaistos Disk, and so on… each typically a piece of ciphertext which we would like to decipher in order to crack a historical mystery. However, one of the items on her list stands out as something of an exception.

For John F. Byrne’s 1918 “Chaocipher”, we have a description of his device (the prototype fitted in a cigar box, and allegedly contained two wheels with scrambled letters), and a fair few examples of both Chaocipher ciphertext and the matching plaintext. So, the mystery isn’t so much a whodunnit as a howdunnit. Though a small number of people are in on the secret mechanism (Lou Kruh, for one), Byrne himself is long dead: and the details of how his box of tricks worked have never been released into the public domain.

Was Byrne’s Chaocipher truly as unbreakable as he believed, or was it no more than the grand delusion of an inspired cryptographic outsider? This, really, is the mystery here – the everything-or-nothing “hero-or-zero” dramatic tension that makes it a good story. Yet hardly anybody knows about it: whereas “Voynich” gets 242,000 hits, “Chaocipher” only merits 546 hits (i.e. 0.0022% as much).

Well, now you know as well: and if you want to know a little more about its cryptography, I’ve added a Chaocipher page here. But the real site to go to is Moshe Rubin’s “The Chaocipher Clearing House“, which is so new that even Google hasn’t yet found it (Moshe emailed me to tell me about it, thanks!) Exemplary, fascinating, splendid – highly recommended. 🙂

OK, enough of the raw factuality, time for the obligatory historical riff. 🙂

I’m struck by the parallels between John Byrne’s device and Leon Battista Alberti’s cipher wheel. Both men seem to have caught the leading edge of a wave and tried to harness its power for cryptography, and made high-falutin’ claims as to their respective cipher systems’ unbreakability: whereas Alberti’s wave was mathematical abstraction, Byrne’s wave was (very probably) algorithmic computing.

Circa 1920, this was very much in the air: when J. Lyons & Co. hired the mathematician J.R.M. Simmons in 1923, the company was thinking about machines, systems, and operational management: mathematical calculators were absolutely de rigeur for them. The first Enigma machines were constructed in the early 1920s (and used in a commercial environment), and there were doubtless many other broadly similar machines being invented at the same time.

Do I think that there was anything unbreakable in Byrne’s box? No, not really: the real magic in there was most likely a programmatic mindset that was cutting-edge in 1918, but might well look somewhat simplistic nearly a century later. But I could be wrong! 😉

Monsters and the abyss…

by ⋅ Leave a Comment

I’m off for a few days now, so I’ll leave you with a thought from Chapter IV of Nietzsche’s “Beyond Good and Evil”:-

146. He who fights with monsters should be careful lest he thereby become a monster. And if thou gaze long into an abyss, the abyss will also gaze into thee.

The little-mentioned thing about this well-known quotation is that the sayings around it are primarily to do with women: so what is Woman to Nietzsche, a monster or an abyss? And is he advising us not fight with or gaze into Her, lest we become Her or She meets our gaze?

The Voynich Manuscript is no monster to fight with: and it is only an abyss for those who are trying to read Hungarian with a French dictionary. Now look to your bookshelf!

Later!

Edith Sherwood’s anagram cipher…

by ⋅ 31 Comments

A new day brings a new Google Adwords campaign from Edith Sherwood (Edith, please just email me instead, it’ll get the word out far quicker), though this time not promoting another angle on her Leonardo-made-the-Voynich-Manuscript hypothesis… but rather a transposition cipher Voynichese hypothesis. Specifically, she proposes that the Voynich Manuscript may well be Italian written in a simple (i.e. ‘monoalphabetic’) substitution cipher, but also anagrammed to make it difficult to read.

Anagram ciphers have a long (though usually fairly marginal) history: Roger Bacon is widely believed to have used one to hide the recipe for gunpowder (here’s a 2002 post I made on it), though it’s not quite as clear an example as is sometimes claimed. And if you scale that up by a factor of 100, you get the arbitrary horrors of William Romaine Newbold’s anagrammed Voynich ‘decipherment’ *shudder*.

More recently, Philip Neal has wondered whether there might be some kind of letter-sorting anagram cipher at play in the VMs: but acknowledges that this suggestion does suffer from various practical problems. I also pointed out in my book that Leonardo da Vinci and Antonio Averlino (‘Filarete’) both used syllable transposition ciphers, and that in 1467 Alberti mentioned other (now lost) kinds of transposition ciphers: a recent post here discussed the history of transposition ciphers in a little more detail.

So: let’s now look at what Edith Sherwood proposes (which is, at least, a type of cryptography consistent with the VMs’ mid-Quattrocento art history dating, unlike many of the more exotic ciphering systems that have been put forward in the past), and see how far we get…

Though her starting point was the EVA letter assignments (with a few Currier glyphs thrown in), she then finessed the letter-choices slightly to fit in with the pharma plant label examples she picked: and there you have it (apart from H, J, K, Q, X, Y, Z and possibly F, which are all missing). All you’d have to do, then, is to anagram the rest of the text for yourself, sell the book rights, and retire to a sea-breezy Caribbean island.

edith-sherwood-alphabet2

Might Edith Sherwood be onto something with all this? No, not a hope: for example, the letter instance distribution is just plain wrong for Italian, never mind the eight or so missing letters. As with Brumbaugh’s wobbly label-driven decipherment attempts, I somehow doubt you would ever find two plausible adjacent words in the main body of the text. Also: what would a sensible Italian anagram of “qoteedy” (“volteebg”) be?

Her plants are also a little wobbly: soy beans, for example, were only introduced into Europe in the eighteenth century… “galioss” is a bit of a loose fit for galiopsi (not “galiospi”, according to “The Botanical Garden of Padua” on my bookshelf), etc.

As an aside, I rather doubt that she has managed to crack the top line of f116v: “povere leter rimon mist(e) ispero”, “Plain letter reassemble mixed inspire” (in rather crinkly Italian).

All the same, it is a positive step forward, insofar as it indicates that people are now starting to think in terms of Quattrocento dating and the likely presence of non-substitution-cipher mechanisms, both of which are key first steps without which you’ll very probably get nowhere.

Nostradamus quatrain cipher…?

by ⋅ 1 Comment

In retrospect, it all seems bleakly inevitable: that enigmatologists would move on from the lardy Bacon-stuffed margins of Shakespeariana to find new hunting grounds. Personally, I thought Nostradamus scrabbled pretty hard to find rhymes for his verses, but a new book claims these were all just a cover story, and that it was no more than an enciphered journal.

“Rise to Consciousness” (2008) by Michal Deschausses claims to decipher the truth hidden in Nostradamus’ work, allegedly revealing a story about “one strong woman” in the far future who seems to resemble… Michal Deschausses?

I-1
Some publishers will assist with offering the knowledge in this Century. In pure reason they will rush her through it. The bout the year of her death secures vast knowledge through the age and the verses will release facts which people will hardly believe.

Could it really be that Nostradamus’ quatrains (strictly speaking, “vers commun”) enciphered a “multi-lingual” secret message? Or is it simply the case that you can, just as Leo Levitov famously did for the Voynich Manuscript, “decipher” just about any given text-like thing into an entirely parallel “polyglot oral tongue”, a stream of Romance language fragments from which you can read basically any message you like?

For reference, Nostradamus’ Century 1 Quatrain 1 is probably an “evocation of the Delphic Oracle, after Iamblichus’s De Mysteriis Aegyptiorum“: and goes like this:-

Estant assis de nuict secret estude,
Seul reposé sus la selle d’aerain:
Flambe exigue sortant de solitude,
Faict prosperer qui n’est à croire vain.

The two are connected, errrm, how? Anyhoo, because Michal Deschausses has been good enough to include her deciphering methodology as an appendix, I’ll probably end up buying a copy and having a look. But having seen Levitov’s book, I won’t be holding my breath.

Now… we’ve had hundreds of years of people claiming that Nostradamus’ verses predict every d&mn thing in the news, from the end of the world, to war(s) in Iraq, and even (most recently) to Barack Obama and his running mate:-

Born of obscure and dark family,
Of white and black of the two intermixed.
The dark one biding his time,
Before the Empire changes.

But… hold on a minute: “Born of obscure and dark family” is a line from Century V Quatrain 84, “Of white and black of the two intermixed” is a line from Century VI Quatrain 10, “The dark one biding his time” appears just to have been made up, while “Before the Empire changes” is a line from Century I Quatrain 43.

So, Nostradamus bloggers have supposedly got themselves excited about three lines moved waaaaay out of context with a filler line added in to form a supposed verse about Obama. Oh, really?

OK… let me try for myself, restricting the search to a high-speed sleective scan through Century I:-

Lost, found, hidden for so long an age,
In the future by headless idiots
That which shall live without having any sense
submerged, killed, dead through brainless idiocy

Profoundly predictive, don’t you think? 😮

What is a ciphertext?

by ⋅ 9 Comments

Essentially, a ciphertext is a piece of text where the individual letters have been transformed according to a rule system – substitution cipher rules replace the shape of the letters (as if you had just changed the font), while transposition cipher rules manipulate the order of the letters.

THIS IS A CIPHER —> UIKT KT B DKQIFS  (substitute each letter with the one after it in the alphabet)

THIS IS A CIPHER —> SIHT SI A REHPIC (transpose the letters, writing each word back-to-front)

So, as long as (a) you know [or can crack!] the rules by which the “plaintext” (the original unenciphered text) was transformed, and (b) those rules can be played out in reverse, then you can decipher the ciphertext.

OK so far… but if you’re looking at historical ciphers, there’s a problem.

Prior to 1400, transposition ciphers were extremely rare, partly because words themselves were rare. Many documents were written without spaces – and without spaces, where do words begin and end? Effectively, this meant that in-word transposition ciphers (such as reversing syllables, as the Florentines Antonio Averlino and Leonardo da Vinci both used) would only happen in those few places (such as Florence) where people had a modern concept of what words were. A well-known modern example is “Pig Latin“, a (20th century) humorous in-word transposition cipher: and there’s the 19th century “loucherbem” in French, too.

Round about 1465, these flowered into some kind of complex system (by an unknown practitioner, and now apparently lost forever): Alberti, writing in Rome during 1465-1467, mentioned a number of ideas for a complex transposition system, though he recommended his own cipher wheel in preference to them.

Yet after 1500, these basically disappeared into the historical footnotes of cryptographic works. What replaced them (circa 1550) was the “rail-fence” Renaissance notion of transposition cipher: this was instead grounded in the print-centric culture of movable type. This saw messages as sequences of characters tick-tocking away to a metronomic beat (i.e. one per tick), and transposition ciphers not as a way of disrupting word contents, but instead as a way of disrupting (& subverting) the metronomic pulse of letters – a very different beast indeed.

THIS IS A CIPHER --> ISTHAY ISYAY AYAY IPHERCAY   (Pig Latin cipher)

THIS IS A CIPHER ---> T I I A I H R    (Railfence cipher)
                      H S S C P E X

It is this latter (16th century) two-dimensional transposition cipher that is widely used in modern cipher-systems, not the late medieval ‘anagrammatical’ transposition cipher.

cipher-timeline

Older histories of cryptography tended to situate all these cipher techniques within what I call a  “progressivist mythology” – the mistaken notion that every new idea not only flows out of all previous ideas, but also improves and refines them. In practice, of course, that’s not how things work : many brief local flowerings of ideas (basically, all the cipher varieties marked in italic above) made almost no impression on contemporary cryptographic practice. Even Vigenère’s autokey cipher (taught on every modern cipher course) did not get picked up by cryptography practitioners for more than two hundred years!

And now for the punchline of this post: if you discard the progressivist mythology, the range of possible local enciphering strategies for a given ciphertext is sharply constrained by the date and position of a document.

I argue that the Voynich Manuscript ciphertext is likely a prime example of this: its internal evidence dates it no earlier than 1450 and no later than 1470 – right at the time of the brief flowering of the kind of syllabic and interline transposition ciphers mentioned by Alberti in his De Componendis Cyfris (1467).

And so, if we seek to apply “pure” modern substitution cipher analytical techniques to something built around an unknown transposition cipher system, we would surely fail to make any sense of it – and this is, I believe, what has happened in the case of the VMs… why it has remained a “cipher mystery” for so long.

“Historian” the 7th best job?

by ⋅ Leave a Comment

Here’s an odd little thing: a site ranking 200 different jobs. What I found interesting there was the complete lack of overlap between the top ten “best” jobs (based on a combination of “Stress, Work Environment, Physical Demands, Income and Outlook”)…

(1) Mathematician, (2) Actuary, (3) Statistician, (4) Biologist, (5) Software Engineer, (6) Computer Systems Analyst, (7) Historian, (8) Sociologist, (9) Industrial Designer, (10) Accountant.

…and the ten most “satisfying” jobs…

(1) Clergy, (2) Physical Therapist, (3) Firefighter, (4) School Principal, (5) Artist (Fine Art), (6) Teacher, (7) Author, (8) Psychologist, (9) Special Education Teacher, (10) Construction Machinery Operator.

I’ve marked in bold those hats which I wear most days (although I’m sometimes accused of being too “preachy” about the VMs, I don’t think I could claim to be a member of a Voynich “clergy”) – 6/10 from the first list, and 1/10 from the second. Curiously, though, I found “Author” to be just about the least satisfying job of all: far too obsessive and antisocial while writing, and more brickbats than bouquets afterwards. 😮

Also, I really wouldn’t have predicted “Historian” would be one of the top 10 highest-rated jobs: but perhaps part of the reason for the enduring level of interest in the VMs is that it appeals to affluent, clever people in good jobs who have leisure time to waste how they please. 🙂

Sarah Goslee’s reconstructed codex…

by ⋅ 5 Comments

A few years ago, Sarah Goslee (who I believe has her own blog here) gradually become more and more interested in medieval / Renaissance history, specifically (in accordance with her science background) with cosmology, astrology, botany, and cryptography. I doubt any Cipher Mysteries regular would be hugely surprised to find out that, somewhere along in the way, she ended up “hooked” on the Voynich Manuscript. 🙂

Her VMs research has mainly concentrated on PCA (Principle Coordinate Analysis) of the VMs’ text: which I think is a bit of a shame, given that the text was apparently constructed in an anti-analytical way to render that kind of approach largely useless. Oh well!

However, infinitii recently emailed me (thanks!) with a link to Sarah’s fascinating description of the late medieval manuscript simulacrum she constructed. Inspired by a fifteenth century Italian herbal and a fifteenth century Austrian alchemy notebook (MS LJS419 and MS LJS382), she set out to create her own SCA (Society for Creative Anachronism)-style astronomical notebook. Structurally, this has limp vellum binding, rag paper, oak gall ink, quill pens, and writing patterned after a fifteenth century herbal: contents-wise, it has a calendar, the metonic cycle, Domenical letters, location of the sun, etc.

It’s a nice, brief description of a well-contained project: recommended! 🙂

Apollo vs Dionysus, the eternal grudge-match…

by ⋅ Leave a Comment

Here’s another post inspired by the book I’m currently reading, Joscelyn Godwin’s “The Pagan Dream of the Renaissance”.

Whereas 15th century Renaissance art was largely orderly, linear, a lot of Mannerist late 16th-century art is disorganized, curvilinear, riotous – this has led to the label of antirinascimento, the “Anti-Renaissance”. But to someone like Godwin with both feet in the iconological trenches, this speaks of a deeper dichotomy – between the ordered Apollonian meme and the disordered Dionysian meme. Godwin pitches the austere, Roman-loving Quattrocento humanists’ dry perspective against the carnal obsessions and pagan thematics of corrupted Cinquecento cardinals – an extended Apollo vs Dionysus grudge-match in an art historical arena.

All of which is quite cool, in an iconological sort of way. 🙂

But once you start looking at things in this way, you begin to see echoes everywhere: in my own research area of Quattrocento ciphers, you could view Alberti’s über-ordered cipher wheel as a quintessentially Apollonian solar device, and then compare it with the apparently disordered, fragmented Voynich manuscript cipher statistics (that I link with Antonio Averlino, AKA Filarete) – Roman austerity against Greek cunning.

Yet does this kind of dichotomistic model really give us a real insight into the kind of secret history that iconologists believe lurks just beneath? Or is it just a modern quasi-thermodynamic meta-narrative (historicizing the universe’s eternal battle of order vs disorder) being stamped over the top of something that is no more significant than a difference in personality?

Reading anything to do with iconography makes me feel like I’m watching a renegade episode of the X-Files, where Mulder and Scully are arguing the toss over something foolishly marginal. Though occasionally I have brief moments where I think “Yes, that does make sense”, you simply cannot infer from the existence of a debate that any of the mad theories being proposed has to be correct. Oh well!

Update: Dennis Stallings points out off-list that the Apollo vs Dionysus grudge-match as an art-historical thema only really kicked off with Nietzsche’s (1872) “The Birth of Tragedy”, which is entirely true – here’s a nice 1996 paper showing (basically) how you can use the A. vs D. dichotomy as a way of blagging your way through literature studies. 🙂

Cryptography vs Cryptanalysis vs Cryptology…

by ⋅ 5 Comments

There are many different ways of, well, reading the unreadable: what isn’t so well-known is that the technical terminology we use tends to highlight those particular aspects that we think are worthy of study (as well as to occult those aspects we are not so interested in). The big three buzzwords are:-

  • Cryptography: writing hidden messages – a historical / forensic approach
  • Cryptanalysis: analysing hidden messages – a statistical / analytical approach
  • Cryptology: reading hidden messages – a linguistic / code-breaking approach

Generally, you’ll see these terms used extremely loosely (if not interchangeably): but that’s something of a tragedy, as each strand is concerned with a different type of discourse, a different type of truth to help us get to the end-line, that of finding out what happened.

(1) If you study the cryptography of the Voynich Manuscript, you would primarily focus on issues such as: the intellectual history behind (and embedded within) the glyphs, the forensic layering of the writing itself, the physical strokes that make up the letters, what corrections there are to be found, how Voynichese practice evolved during the construction of the document, how the writing interrelates with the drawings, etc. This is reconstructive forensic history, that seeks to establish the truth of the writing system – to establish the mental structures that were given systematic shape (and yet were hidden) in the writing. In many ways, the end-product would be an accurate transcription of the text – but I strongly believe that this strand has not yet been pursued to its logical conclusion.

(2) If you study the cryptanalysis of the Voynich manuscript, you would instead take the study of the cryptography completely as a given, and use the resulting transcription as a starting point for your analytical research, however (in)accurate it may be. The argument has typically been that even if, say, 10% of the transcription is wrong, statistical analysis of the remaining 90% should still yield informative results that are (to a certain degree) illustrative of the underlying mechanisms. Yet the specific reliance upon the transcription cannot be ignored, particularly when you go hunting for larger-scale patterns (such as words, or lines).  And there is a very strong case to be made that the absence of convincing statistical results to date arises not from inadequate statistical testing, but instead from some basic division within the text being misunderstood.

(3) If you study the cryptology of the Voynich Manuscript, then you would take as a given a carefully-selected set of statistical properties previously derived from cryptanalysis, and look for some kind of linguistic fit between those properties and the properties of known languages and/or transformations of known languages (such as shorthand, patois, abbreviation, contraction, etc). Many Voynich theories are based on a very naive cryptological reading, often filling the vast gaps between the two models by expanding the range of possible languages that are present all at the same time, and hence resulting in a claimed plaintext that is a hugely interpretative soup of Romance language fragments – though Leo Levitov’s “polyglot oral tongue” is a prime example, it is very far from being the only one of its kind.

In terms of this framework, I’ve invested most of my time on the VMs’ cryptography, to the point where I believe I can give an account of each of the glyphs and of the evolution of the writing system: but I’m now at the point where I have to move on to the cryptanalysis in a more focused way to make progress.

The overall point I’m trying to make is that we need to get the history (cryptography), the statistics (cryptanalysis) and the linguistics (cryptology) sorted out in order to get over the high walls of the Voynich Manuscript’s defences: its singular beauty arises from how it manages to confound all three of these approaches simultaneously. This is, I suspect, merely a byproduct of the ‘undivided’ Quattrocento thinking that gave it life – that it comes from the time-period just before we (as a culture) imposed artificial divisions on the way we think about the world… just before intellectual specialization took hold. The historian part of me wants to shout: look, it’s the product of a Renaissance Man, in every useful sense of that much-abused phrase.

The 300th post: whither goest 2009?

by ⋅ 4 Comments

“Cipher Mysteries” blog statistics: 300 posts, 11 pages, 1000+ spam comments, PageRank 3 home-page, 41 readers (via FeedBurner) and 15,000+ visitors. Thank you all for the 181 on-blog comments and the hundreds of off-blog emails I’ve received: these really help make this whole thing worthwhile! 🙂

And thanks to an extra 600-visitor surge over two days (from an unknown US-based mailing list’s link to a Stumbleupon link), the blog had more than 3000 visitors during the last month: at the current rate of growth, it should get 75,000 visitors by the end of 2009 (which would be nice).

I’ve also recently started rebuilding the site infrastructure, by moving the stats over from SiteMeter to StatCounter (which has a better API, better reporting and no tracking cookies, though how you make .htaccess allow the StatCounter .js file to be “Accept-Encoding: gzip” I don’t yet know), and by modernizing the icons & transforming them into CSS sprites. Unfortunately, I then got tangled up with irritating browser-related CSS sprite issues. Even so, blog pages are now about half the size they were before and get served up much quicker, which is rather pleasant. 🙂

The main web-tools I used to achieve this were: (1) a free web page speed analysis tool from WebSiteOptimization.com (very handy for blogs with multiple plugins!); (2) a very nice CSS sprite generator on website-performance.org; (3) the CSS Compress WordPress plugin which (very handily) gzips your blog CSS files; and (4) the WP Super Cache plugin, which is (unsurprisingly) a super-duper HTML cache for WordPress. All of which I highly recommend! 🙂

But enough of the blogophile jargon-fest: what can I glimpse looming for 2009 in my polished obsidian mirror? Whither goest the next 100 posts?

Whereas 2008 was (as predicted) the year of the Voynich novel, and 2010 looks to be the year that the Voynich enters the academic mainstream, 2009 looks to me very much as though it is going to be an odd, transitional sort of year – a period of behind-the-scenes activity, which astrologers would normally recognize as a “12th house” (just below the horizon, shortly to rise with the ascendant) kind of vibe. In a strange way, it feels to me as if a future king/queen is preparing his/her entrance on the scene – as if all we have been doing is tamping the road surface for them to drive over it at great speed. Sorry: as predictions go, that’s as close to Nostradamus as I get. 🙂

Regardless, I look forward to being pleasantly surprised by whatever transpires in 2009, and I hope it turns out to be entertaining and interesting for you too! 🙂