Happy Humbug!

Evidence-free rants and their perpetrators

In 1968 my family supported the Humphrey-Muskie ticket against Nixon-Agnew. My parents had not yet lost their minds to right-wing nonsense. Back then, Nixon was political evil incarnate. (Today, Dad dismisses Nixon's transgressions—subverting the nation's electoral process—as trivialities compared to Clinton's dalliance with Lewinsky or Obama's health-care reform “death panels.”) Even back then, as far as Dad was concerned, it didn't do to wear one's political heart on one's sleeve. He promptly peeled off the Humphrey-Muskie bumpersticker that I had affixed to the family car. He disapproved of stickers on cars.

Imagine my surprise, therefore, when last year Dad applied a “We Say Merry Christmas” sticker to his vehicle. I guess the war on Christmas is an occasion even more fraught with peril than a Nixon presidency. You have to admire the sentiment, too: A Christian refusing to turn the other cheek by hurling “Merry Christmas” in the faces of people who dare to say “Happy Holidays” or “Season's Greetings.” It's time to take it to the enemy!

The pseudo-intellectual talk-show host Dennis Prager was holding forth on his program this morning on the importance of public observation of Christmas in this Christian nation. As a practicing Jew, Prager brings a special éclat to the mistimed celebration of the birth of baby Jesus. It must surely delight Prager's arch-conservative Christian listeners to hear him endorse their positions, even as it must occasionally taint their joy slightly to think that he will surely burn in hell one day.

I call Prager a pseudo-intellectual because I have on a number of occasions heard his largely evidence-free pronouncements on the great issues of the day. In fact, he glories in his self-proclaimed erudition as he presents his “Ultimate Issues” hours, during which he panders to the prejudices and preconceptions of his audience. He makes them feel smug and happy because such a well-spoken smarty-pants agrees with them. But his intellectual underpinnings are built on sand. For example, Prager has swallowed Intelligent Design creationism whole. He likes arguments based on personal incredulity and he can't imagine life occurring without God to create it and guide its development.

Therefore I was less than impressed when Prager lamented the death of “Merry Christmas” as a holiday greeting. He declared, with great assurance, that pressure from anti-religious pressure groups had brought nonsectarian greetings like “Happy Holidays” into prominence in preference to speaking of our (not his) dear savior's birth. Instead of taking Prager's word for it, I decided to do a little checking. What does Google's Ngram viewer show?

If one searches through Google's textual database for American English publications between 1900 and 2008, we discover that writers in the United States have favored Happy Holidays by a wide margin over Merry Christmas. (Season's Greetings is a sorry also-ran.) Only twice has the primacy of Happy Holidays been threatened: the era of World War II and the period of the Vietnam War. Both lengthy conflicts coincided with an upsurge in the use of Merry Christmas. (One wonders why.) Of course, it may be that spoken greetings were entirely at variance with authorial word choices during the past century, but I think it's rather more reasonable to expect some parallels. Certainly this tends to run contrary to Prager's claim.

I'm sure, however, that Dennis Prager could find something to like in this data—as unused as he is to looking at any—by homing in on the rise in Happy Holidays in recent years. He could put his finger on 2003 and say, “See, this is the war on Christmas taking hold!” (It would be impolite to point out that Merry Christmas also experienced a spike. In fact, in relative terms its growth is even greater. But that would spoil the narrative.)

If Dennis wants a slam-bang finish, I'll point out that at the end of the chart, in 2008, just as Obama was winning the presidency, Happy Holidays reached its highest peak since about 1917, just as Lenin seized control in Russia with the Bolshevik Revolution. Coincidence? Impossible!

Read more...

And still counting...

Predicting the future

The 2010 general election is not yet over in California. The secretary of state's office in Sacramento continues to issue updates as it aggregates the returns trickling in from California's fifty-eight counties. As of the last report, time-stamped 5:00 p.m. on Friday, November 19, twenty-six counties still had untallied ballots to count. By adding up the counties' estimates of vote-by-mail and provisional ballots, the secretary of state announced that approximately 629,634 votes remained to be processed.

It's not a moot point. While attorney general Jerry Brown trounced Meg Whitman in winning his third term as governor and Barbara Boxer convincingly defeated Carly Fiorina on her way back to the U.S. senate, one statewide race remains too close to call. The Democratic nominee to succeed Brown as attorney general currently has 4,291,854 votes to her Republican rival's 4,248,804, a margin of only 43,050. Taking into account a scattering of votes among minor party candidates, that breaks down to 46.0% to 45.5%.

Democrat Kamala Harris and Republican Steve Cooley have swapped the lead back and forth a few times since the vote tallies began to be published after the November 2 election. Cooley actually declared victory election night (see the video below), but woke up the next morning to discover that Harris had edged ahead. When the vote count moved him back into the lead a few days later, he was smart enough not to make yet another premature victory speech. When Harris regained the lead, she prudently kept her own counsel.

My own opinion? For a couple of weeks now, I have been expecting a Harris victory. It's in the numbers.

I looked into the numbers because one of my friends, a retired journalist, was scoffing at the superficiality of the news articles on the election results in the attorney general's race. Except for striving heroically for different ways of saying “too close to call,” none of them offered any substantive analysis.

“It all depends of where the remaining votes are,” he said. “Instead of just paraphrasing the press releases from the candidates, the reporters ought to dig into the details. They should do some reporting.”

He prodded me into action. I downloaded the secretary of state's report on unprocessed ballots (well over two million at that time) and loaded it into a spreadsheet. Then I perused the secretary of state's report on the percentages accruing to each candidate in each county. By way of example, consider Tulare county, where California's most conservative voters gave Harris only 29.8% to Cooley's 62.4%. Tulare's county clerk estimated that 3,350 ballots remained to be processed. Applying the percentages to this number, I computed that Harris would get 998 more votes and Cooley would get 2,090. (I'm sure Mom & Dad's vote-by-mail ballots are in the latter batch.)

I applied this process to all of the counties with outstanding ballots, obtaining an estimate for the additional votes likely to be obtained by Harris and Cooley. Upon adding the estimates to the votes counted to date, I found myself looking at a razor-thin Harris victory. Every so often I would return to the secretary of state's website to tweak the percentages to reflect the completed count. Those numbers were very stable, seldom moving more than one-tenth of a percent. The predicted Harris margin varied, but never vanished.

My latest computation, based on yesterday's numbers, suggests that Kamala Harris will defeat Steve Cooley for the office of attorney general by 45,902 votes. I'm not sure about the 2, though.

If the numbers hold up, the Golden State will have handed the Democratic Party a clean sweep of every statewide office. May it make the most of its opportunity.

---

Note: I should give a tip of the hat to Timm Herdt of the Ventura County Star. He had the same idea that I did and published his estimate on November 9 on his blog. In my opinion, however, Herdt pulled up just a bit short by confining his attention to the 21 counties with the most votes remaining to be processed. In so close a contest, it was unwise to scorn the little counties and risk that much round-off error. On the basis of his computations, Herdt figured that Cooley had an edge.

While I obviously think Herdt was wrong, my ex-journalist friend can be relieved to learn that at least one reporter is willing to go digging for news. It's not quite obsolete yet.

---

Read more...

Laughing at what you don't understand

Entertainment at tea time

The anti-intellectual culture of the day must be quite a treat for the subomegaloids who watch Fox News and frequent Tea Party gatherings. One of the country's major political powers is now a wholly-owned subsidiary of the know-nothing right wing, indicating a kind of high-water mark for bumptious ignorance and anti-science. Still, it remains something of a disappointment when the waves slop over onto New York magazine. You'd think that a publication naming itself after the cosmopolitan Big Apple would keep a stiff upper lip and refuse to frolic among the scum-sucking bottom-dwellers.

Well, you'd be wrong.

This past week New York magazine decided to go for some easy yuks by mocking some math classes. It was a piece of cake. A group of five people—yes, it took five people to do this—pored over the course offerings at several American liberal arts colleges, found some math courses they didn't understand, and merrily made fun of them.

What a treat for New York's readers. (The ignorant ones, at least.)

Let's join the fun, shall we?

The Ten Most Ridiculous-Sounding Math Classes Currently Offered at Liberal-Arts Colleges

October 19, 2010

Every year, liberal-arts majors anxiously scour their college's course listings looking for classes that will fulfill their math requirement but aren't so, you know, math-y. Here's what they're signed up for this year.

10. Topology: The Nature of Shape and Space: “In geometry we ask: How big is it? How long is it? But in topology we ask: Is it connected? Is it compact? Does it have holes?” [Sarah Lawrence]

Topology is not a trivial topic. By what metric do the writers gauge this to be a “ridiculous-sounding” course? They're off to an embarrassing start (assuming they're capable of embarrassment).

9. The Mathematics of Chance: “Most topics are introduced in a case-study fashion, usually by reading an article in a current periodical such as the New York Times.” [Bard]

And already it gets worse. Probability and statistics pervade our technological culture but are often misunderstood. If people had better number sense, they would be fooled less often by nostrum-peddlers and dishonest politicians. Scanning a periodical is an excellent way to find numerical arguments (e.g., polling data, unemployment figures, medical claims) that should be subjected to some critical thinking.

8. Mathematics in Many Cultures: “Mathematical ideas are found in many cultures, among both literate and non-literate peoples. This course examines both mathematics and the role it plays in the cultures. Examples chosen from the mathematical ideas of present-day peoples of Africa, Asia, Oceania and the Americas, as well as historic Egypt, Mesopotamia, Greece, Islam and China.” [Pomona]

Mathematics is a human endeavor that spans cultures and exhibits the variety of our thinking. The course title is straightforward and the description apt, yet the New York contributors deemed this another “ridiculous-sounding” math class.

7. The Magic of Numbers: “This course will explore the beauty and mystery of mathematics through a study of the patterns and properties of the natural numbers 1, 2, [and] 3." [Harvard]

I clicked through to the course link on this one. Sure enough, the writers had “improved” the course description by omitting the ellipsis after the numerals. (The natural numbers go on forever. And what does Harvard say about this “ridiculous” course? Here's the rest of the description: “We discuss various special classes of numbers, such as prime numbers, factorials, and binomial coefficients, and the many ways they arise in mathematics. We will discuss questions in probability (such as: the likelihood that two people in a class of 25 have the same birthday). We also study modular arithmetic and secret codes based on it.”

Yeah. Sounds stupid, doesn't it?

6. Models of Life: “In particular, we will ask such questions as: How do you model the growth of a population of animals? How can you model the growth of a tree? How do sunflowers and seashells grow?” [Kenyon]

This terse description is sufficient to identify exponential growth as one of the topics (for population, naturally) and Fibonacci sequences for sunflower growth (and other cases, too), and probably the over-hyped but still nontrivial golden ratio.

So inane. The New York contributors must have dislocated their jaws from yawning (when they weren't sniggering, of course).

5. Mathematical Origamist’s Toolkit: “Topics include modular origami and how this models the creation of polyhedra and coloring of graphs, comparison of origami-axiomatic constructions to straight-edge and compass constructions, the combinatorics of possible crease patterns, the mathematics of origami design (circle packing, optimization), matrix models for paperfolding, spherical geometry, Descartes’ Theorem, and Gaussian curvature.” [Hampshire]

I can think of only one possible reason for the inclusion of this course: The writers (and I use the term loosely) thought that “origamist” was just too funny for words. This course is full of clever and subtle stuff that a good instructor could have a lot of fun with—and I don't mean the kind of mocking fun that New York magazine is trying to have at its expense.

4. Mathematics and Narrative: “Many literary works (Arcadia, Proof, and Uncle Petros and the Goldbach Conjecture) use mathematics as an integral part of their narrative. Movie and television narratives such as Good Will Hunting and Numb3rs are also mathematically based. Nonfiction works about mathematics and mathematical biographies like Chaos, Fermat's Enigma, and A Beautiful Mind provide further examples of the connection between mathematics and narrative.” [Vassar]

Sir Tom Stoppard? David Auburn? Apostolos Doxiadis? Heck, no one has ever heard of these writers. Right? At least, the New York writers appear to have failed to see the value of exploring the mathematical underpinnings of major recent literary works. Why, they didn't even appreciate Good Will Hunting, which was a movie they could watch in slack-jawed amusement while munching popcorn (while ogling Matt Damon). I doubt, however, that they would appreciate the accuracy of the mathematical boardwork that appeared in the movie. (I did!)

3. Borges and Mathematics: “Jorge Luis Borges was one of the most important writers of the 20th century. Many of his short stories and essays were concerned with philosophical, metaphysical, and mathematical questions. The thesis motivating this course is that if we know the mathematics that Borges referred to, then we will read him differently, and we will read him better.” [Bennington]

The inclusion of this course is beyond outrageous. The New York magazine ni kulturni don't recognize the significance of mathematics in the work of Borges, who famously imagined “The Library of Babel” and thereby created a fascinating combinatorial icon that has persisted in literature and mathematics.

2. Mathematics and Science as Art in Contemporary Theatre: “Playwrights such as Tom Stoppard, Rinne Groff, Michael Frayn, and others have effectively explored mathematical and scientific themes for artistic purposes. Through readings and exercises, and by conducting labs and staging scenes, this class will gain some first-hand insight into the complementary ways in which science and art aim to seek out their respective truths.” [Middlebury]

Aha! A theme! (And three in a row.) These people don't like any explorations of the connections between lit and math. And they were also getting desperate to pad their list out to ten.

1. Meaning, Math, And Motion: “Quoting a charming articulation by Kinsman (a mathematician-turned-oceanographer, in the preface to Wind Waves): 'To the beginner, science is a conversation that has been in progress for a very long time.' Our collective work is to catch up on the conversation.” [Evergreen]

At this point, my patience is exhausted and I'm unwilling to give the benefit of any doubt to the writers. This one might be just a little light-weight and silly, but I've come to regard their disdain as a mark of distinction. Go ahead and take the class, students of Evergreen College. It's probably better than its rather vapid description.

The list of “ridiculous-sounding” courses ends with a bit of snark:

This is why Asia is winning, by the way.

I have a better suggestion, but the New York contributors won't like it (even though they consider mockery an art form).

Read more...

Believing your lying eyes

One picture is worth a thousand misstatements

PZ Myers over at Pharyngula is more pleased than he should be with some “flashy illustrations” of the creationism menace. “Use these!” he admonishes. Um, maybe not.

While the graphs are flashy and, in their way, are informative (though I'm not crazy about the choice of pale red [is not explicitly mentioned] as intermediate between green [is mentioned directly] and dark red [isn't mentioned]—pick a better color palette, guys!), there are significant scaling problems. When one ostensibly represents ratio data with the linear dimensions of objects possessing area, the visual impact is seriously misleading. Perhaps this sounds fancy, but it's not.

Suppose you want to compare a data value of 16% with a data value of 32%. The latter is twice as much as the former. In a bar chart, one would be twice as tall as the other. Simple:

You can't get much more basic than that. However, what if you decide to represent your data with cute little gingerbread men? It's quite obvious that the one representing 32% must be twice as tall as the one representing 16%, right? Except look at what happens:

It's the classic problem of scaling. Doubling the linear dimensions of a two-dimensional figure results in a quadrupling of its area, immediately creating an exceedingly misleading visual impact. To maintain a correct visual impression in a situation where two-dimensional area rather than one-dimensional length catches the eye, the linear scaling factor should be the square root of 2 (approximately 1.4142) rather than 2 itself. The results are much better:

This is exactly the problem with the Campus Explorer's iconic graph of teachers' personal beliefs in evolution vs. creationism and intelligent design creationism. The purple figure representing 28% looks like it's quite a bit more than twice the 16% icon. The 16% icon similarly looks a lot bigger relative to the 9% icon than it should. The Discovery Institute will certainly be delighted with the colossus representing adherence to ID creationism.

With some elementary (and not particularly elegant) picture editing, I offer this statistically improved version, whose visual impact is not misleading:

Darrell Huff warned us about misleading data graphs in How to Lie with Statistics, originally published in 1954. We have yet to learn the lesson he tried to teach us.

Read more...

Counting cows

Accidentally in glut

People have been bringing a recent New York Times article to my attention: “From Science, Plenty of Cows but Little Profit.” (If the New York Times is behind a subscription wall for you, try the News & Observer's reprint here.) Just when milk prices crash and dairy farmers are taking it on the chin, science has foolishly tampered with nature and created more cows! As reporter William Neuman explains,

Three years ago, a technological breakthrough gave dairy farmers the chance to bend a basic rule of nature: no longer would their cows have to give birth to equal numbers of female and male offspring. Instead, using a high-technology method to sort the sperm of dairy bulls, they could produce mostly female calves to be raised into profitable milk producers.

Now the first cows bred with that technology, tens of thousands of them, are entering milking herds across the country—and the timing could hardly be worse.

There's less to this story than meets the eye.

It's true, of course, that any dairy farmer who invested in sex-selection technology wasted his money. Money he almost certainly cannot afford in these straitened times. The extra heifers are of no benefit, since he certainly does not need more milk cattle while the market for fluid milk is glutted. Dairy farmers are now receiving about $11 per hundred-weight, while a year ago it was over $19. The dairymen who went into debt to expand their herds at the price peak are now losing those herds at the low point. (My brother, who is a dairy farmer, knows of two men who got out of the business the hard way: one used a rope and the other used a gun.) It's a disaster out there, as depicted recently on ABC World News (“Dairy Farms Disappear”).

The problem, therefore, is real. The sex-selection technology, however, is the merest blip. It might be a significant boon in the future, supposing that there is a recovery in the dairy industry, but in the short run it has minimal impact. You see, each milk cow needs to give birth in order to start lactating. You may remember that fact from basic biology. Perhaps at some point we will overcome that limitation, but for now it still holds true.

This basic law of milk production means that each cow in a milking herd is a mother, implying the existence of a calf. The number of calves isn't going to change. In the past, you had a fifty-fifty split in the number of heifers (female calves) and bull calves (male calves). As reported in the New York Times,

The male calves are usually sold for little money to be raised as meat, and the females are raised as milk producers.

All too true. The occasionally bull calf may be raised to adulthood for stud services, but most of the boys go right to the auction block (along with an excess girl or two).


Such a waste of breeding time and effort, producing all those useless males!

It looked for a moment like the problem of excess males had been fixed. Unhappily, under current circumstances, the fix merely means that we have begun producing useless females—in a one-to-one substitution for the useless males. The excess heifers will be sold off instead of being bred and joining the milk herd. The unneeded heifers will command the same meagre meat-product prices as the unneeded bull calves. The girls will, unfortunately, have cost more than the boys used to because of the investment in sex-sorted semen for artificial breeding purposes, but they won't be adding a single drop to the milk glut.

They'll never be milked.

Read more...

Ain't it the truth!

Outliers are people, too

The University of California called me up and we had a nice chat the other day. Perhaps “chat” isn't the right word. The UC asked me lots and lots of questions and I gave lots and lots of one-word answers. I am, you see, a research subject.

Or was, anyway. I think we're done now. I'm not completely certain how I came to the UC's attention. The letter announcing their interest in me said I was randomly selected for their research pool, but it wasn't clear to me whether they went sifting through their alumni database, the phone book, or names scrawled on the walls of campus restrooms. (I'm guessing it wasn't the latter.) Frankly, I don't have much doubt about the ability of the researchers at the ten University of California campuses to work up reasonably representative samples for their investigations. I believe they probably did a good job in picking their subjects for this research project as well. And we all understand that every large random sample can be expected to have an extreme outlier or two.

That's me!

I'm sure the researcher on the other end of the phone line caught on fairly quickly. We were just tearing through his list of carefully worded questions, pruning away most of the question tree's branches of follow-up questions because I kept saying “No” to the root query:

“In the past six months, have you had any alcoholic beverages?”

“No.”

“In the past six months, have you smoked any tobacco products?”

“No.”

“In the past six months, have you used any nonprescription drugs for recreational purposes?”

“No.”

“In the past six months, have you ever smoked marijuana?”

“No.”

“In the past six months, have you used any recreational drugs like Ecstasy?”

“No.”

“How easy would you say it is to obtain marijuana in your community? Very easy, fairly easy, somewhat difficult, or very difficult?”

“Hey, your Likert scale doesn't have a midpoint answer! Was that deliberate?”

[Laughs] “Uh, shall I repeat the question?”

“Well, I guess you could, but the real answer is that I have no idea. I mean, I live in a college community so I suppose it's not that difficult to find some pot, but I haven't a clue where to begin. Maybe I could go chat up some colleagues in the English department. The poets, anyway. But I think their drug of choice is mass quantities of alcohol.”

[Laughs] “Okay. I think this is a ‘don't know’ response.”

It's true. I have not the faintest idea how to “score” some pot around here. (I don't even know if people are still using “score” in that way.) For some reason (and I really don't know what the reason is), I have never been tempted by drugs or alcohol. Unlike my kid brother, I never swiped a cigarette from my grandfather's pack of Winstons just to try it out. (Dad sniffed out the discarded butt in the trash can and my brother got a walloping. I was not even a suspect.) While my sibs liked to suck out the last few drops from Dad's beer cans, I ignored them. I've never had a puff from a cigarette and I've never really had a drink of alcohol (unless you insist on counting a perfunctory sip of champagne during celebratory toasts at weddings or drops of wine (blood!) from an earthenware chalice at the trendy pass-it-around communion ceremony at Catholic Newman Center services in the seventies). Yes, I've made it to middle age without ever having been drunk or stoned.

Of course, it doesn't do to make a big deal of all this. And it's hardly a virtue to refrain from that which does not tempt you. It merely mystifies the addicts among one's acquaintances—those who cannot imagine living without their periodic alcoholic buzzes or nicotine rushes. So I don't say anything, unless a debate arises over who has to be the designated driver, in which case I casually volunteer. I'm noble that way. I wear my faux virtue lightly.

Sexy little numbers

The UC researcher was quite good at dealing with me. He never once made even the slightest comment that my responses were several standard deviations away from the mean. He must have discerned that I had some familiarity with research protocols (especially after he asked the questions about my education level), but he was scrupulous not to break character and indulge in any wink-wink entre-nous witticisms. I was impressed by his professionalism. I never got more than a chuckle out of him, or perhaps just a slice of wry in one of his noncommittal responses to one of my remarks. I laughed aloud after he asked three consecutive questions about sexual relations:

“How many sex partners have you had in your lifetime?”
“How many of your sex partners were female?”
“How many of your sex partners were male?”

“Hey,” I said, “I'll bet the second and third numbers are supposed to add up to the first number!” We mathematicians are always on the alert for that. (And, no, it has nothing to do with the Law of Large Numbers.)

“There are certain redundancies in the questions for internal validation,” the UC researcher admitted dryly. He passed up the opportunity to mention the possibility of transgender, hermaphroditic, intersexed, or extraterrestrial partners. Probably just as well. (“Who do you find more sexually attractive, Kodos, Kang, Leela, or Roger?”) Certainly that could throw off the numbers. For example:

“How many sex partners have you had in your lifetime?”

“Thirteen.”

“How many of your sex partners were female?”

“Seven.”

“How many of your sex partners were male?”

“Two.”

“Huh?”

No stone unturned

As best as I can recall, I have seen exactly one marijuana cigarette (reefer madness!) in my entire life so far. It was back in the seventies during a gathering of friends and acquaintances from my hometown high school and junior college. To my gobsmacked surprise (although I hope I hid my reaction), a friend and classmate from my calculus class matter-of-factly laid out a cigarette paper on the kitchen table, sprinkled some shredded brown leaves on it, rolled it up, lit it, and took a quick drag. Nobody turned a hair, so I tried to maintain my cool (such as it was). I was sitting right next to him, so he passed it to me.

I touched it.

And in exactly the same affectless way in which he had performed the entire little ceremony, I nonchalantly handed it off to the next person. I realized that I was actually among friends when absolutely no one ragged on me for merely passing it along. But I was among friends with whom I had even less in common than I had thought.

Well, that was nothing new.

A few years later, at the tail end of the seventies, I did encounter something a little different. I was now at a university. A fellow math grad and I decided to take in as much of a 24-hour movie marathon as we could stand. It was billed as an eclectic mix of standard thrillers and the famously awful (e.g., Bedtime for Bonzo, in which Ronald Reagan supposedly uses gestalt techniques to mind-meld with a chimp, and Sextette, in which Mae West pushes her sex-siren vamping perilously close to necrophilia). My buddy and I laughed and groaned through hours of entertainment and quasi-entertainment, until eventually it was well past midnight and we were still hanging in there.

In retrospect, it was probably inevitable that someone would light up in the darkened theater. Tendrils of sweet, wispy smoke eventually wafted down to where we were sitting. My classmate raised his head and took a couple of audible sniffs. He turned to me and said, ”What's that?” Suddenly I knew that I had a friend for life.

He had earned his bachelor's degree from UC Berkeley and yet could not recognize the odor of pot smoke.

Buddy!

Read more...

Roller-coaster polling

Keep your barf bags handy

I like to follow political polls and I'm encouraged that my preferred presidential candidate continues in most reports to hang on to a significant lead. Nevertheless, I am heartily tired of the constant “analysis” of nonexistent volatility. It's a pseudo-drama ginned up by a combination of deep ignorance and ratings-driven sensation mongering.

It's polling, suckers. It's a statistical art with well-documented error bars. The results are not five-decimal-place scientific calculations. They're probabilistic estimates. The numbers are going to jump up and down even if nothing changes. Got that?

For purposes of illustration, suppose that a poll's accuracy is described as being within 3 percentage points 90% of the time. Got that? Then about 10% of the time it will be off by more than 3 points. If two candidates are within about 5 points of each other, taking 3 points away from the leader and giving them to the other guy will reverse the race! Only not really, because the poll would be in error in that instance. These flukes are not avoidable and they provide grist for the mills of the talking heads who will then scream about upsets and stunning surprises. They're just idiots. Or ignoramuses. Maybe both.

Let's take the illustration a few steps further. We can use a normal distribution (“bell curve”) to model the results of polls whose errors are less than 3 points 90% of the time. With the assistance of Excel's random number generator and built-in normal distribution functions, I ran two dozen trials and plotted the results of the simulated polling errors:


The model calls for the errors to fall within plus-or-minus 3 points 90% of the time (in the long run, mind you). Ten percent of 24 trials is 2.4, so we should expect maybe 2 or 3 results to fall outside the plus-or-minus 3 band. In this little experiment, it happened four times, twice going high and twice low. Please note the dashed line marking the −2.5 boundary. Whenever the leader's margin was underestimated by at least 2.5 points (in a race with an actual 5-point margin), the lead changes hands—according to the poll, not reality. Imagine the breathless reports that would immediately blossom on the cable news networks, newspapers, radio talk shows, and blogs. Panic! Or elation!

Another thing to keep in mind: The model did the exact same randomized calculation each time, assuming no actual change in the electorate. You still get variation, an inevitable consequence of sample-based statistics. You're stuck with it and you have to live with it. And pundits love to live with it because it gives the illusion of motion even when none exists.

People should just calm down.

I swear that I did not cherry-pick my results in order to present one that looked especially dramatic. The graph above shows my initial run. I cranked out some more examples, just to see what they looked like. These appear, together with the original experiment, in the graph below. Does it look chaotic enough for you? (Only the blue graph failed to produce an artificial swap in the lead.) See? Major developments in the campaign! We have headlines!

Read more...