What Information Theory Teaches Writers

Sat May 28 2016

This is Claude Shannon, a man of mythic status within Nerdlandia. Outside Nerdlandia, hardly anybody knows of him.

I won't summarize his work, because Wikipedia already has. Suffice it to say that Claude Shannon figured out information is measured in bits; a staggeringly important fact in our digital world.

His work in information theory is densely mathematical, yet also reveals the reason behind one of writing's most well-known rules of thumb: "Shorter is better." In homage to Shannon, I will make this post as short as I can.

Communication for Nerds

Nerds view communication in terms of messages. Any message--an SMS, a selfie, a song, or a story--contains some amount of information. That's the "signal," in information theory: what you want the recipient to have.

However, we don't dump the raw bits of our message directly into recipients' brains. Messages are always communicated by encoding them according to some scheme. Digital pictures use encoding schemes like "JPG," or "PNG." I am communicating this blog post to you in an encoding scheme called "English."

Signal and Noise

Claude Shannon was deeply interested in the relationship between how many bits are in your message, and the length of that message when you're done encoding it.

Not only can you measure how many bits make up your message, you can also measure how many bits make up the encoded message, and these figures are rarely the same.

What Shannon found was that a perfect encoding scheme wastes no bits. An imperfect one does, which equates to a longer message to be written, stored, transmitted, received, and eventually decoded. All of which is just more work for everybody.

Shannon termed these wasted bits as "noise," and the ratio of signal to noise as a measure of a transmission's quality.

Signal Quality

Think of a radio station you can barely tune in on a long car trip. It's mostly static. You can hear the music if you listen really hard, but it takes a lot of effort to filter out the static and you probably won't enjoy it very much. The transmission has a crappy signal-to-noise ratio.

Contrast that with a new-fangled digital radio transmission: crystal clear, no static at all. It's easy to listen to and far more enjoyable because the transmission has an awesome signal-to-noise ratio.

Signal-to-Noise for Writers

By some estimates, English carries about 10 bits of information per word.

If you edited your story down to its most distilled, essential form, but without removing any story elements—that is, without changing the message—you could easily calculate the information content.

Let's imagine the distilled form is fifty thousand words, or 500,000 bits.

How long is the un-distilled manuscript? Seventy-five thousand? Eighty? Or did you pad it out with repetitions, wordiness, and telling readers things they can figure out for themselves, all the way to the ego-boosting 100,000 word mark?

You did? Congratulations! You just made readers decode a cool million bits of transmission to get the half-million bits of actual message.

Don't expect them to thank you for a paltry signal-to-noise ratio of 1/2.

Shorter is Better

Shorter is better because you make readers work less for the same reward. I've talked about that before for kids, but it's just as true for adults. By giveing readers the same story in fifty thousand words rather than a hundred, they get the same reward for half the work.

In retail, that's called a "sale," and as shoppers we're all looking for bargains.

It's no different in writing. Readers may buy your book with dollars, but they buy the experience of your story with work.

Make your book a bargain. Same signal + less noise = better experience. By keeping it short, your message comes through more clearly.