Mercury Retrograde, Amazon Review Misspellings, and Artificial Intelligence

I gave this talk at the Kepler Conference in January 2017 and posted it here. I have left it as it was. The note below is what I know now.

This is the last of five pieces I wrote about Mercury retrograde and misspelling, and it is the one where I reached for artificial intelligence. I have appended notes to the first and the fourth setting out what is wrong with the design they all share. Here I want to deal with the thing this page is actually about: the machine.

The dataset behind this talk is the one I finally tested

For a while I could not find the data underneath any of these studies — I had published the notebooks and lost the numbers, the recipe without the ingredients. Then I found fullamazonmisspellingdata.csv, and for some time I could not tell which study it belonged to: its rate averages 0.24, where the 2015 Fourier paper's daily averages sit near 0.016.

It belongs to this project. My own deepnet write-up describes it exactly — 5,296 rows, one per sequential day, Jan 1 2000 to Jul 1 2014, right ascensions and retrogressions across the top — and offers the same file, at the same 783 kb, for anyone to download. And the red wave at the top of this page swings between 0.24 and 0.31, centred on 0.27, which is precisely this file's rate. So the tests below are not a distant proxy. They are this talk's own data.

What the file says when you ask it properly

Slide Mercury's retrograde flag to every possible position along this series and ask whether its true alignment stands out from the arbitrary ones. It does not: p = 0.449. Then do the same for every other planet the file flags — and it flags all eight, which I never once checked. Mercury comes fifth of eight. The planet that clears the usual bar is Uranus, at p = 0.012, which is simply what a false positive looks like when you take eight bites at a five percent apple. The working is at the foot of the fourth article.

The machine did learn something. It was not Mercury.

Now the artificial intelligence, which is what this page promised. I gave a model all ten longitudes and all eight retrograde flags and asked it to predict the misspelling rate. It manages an R² of about 0.17 on held-out days, which looks like a discovery.

Then I asked the same features to predict, instead of the misspelling rate, what day it was. R² = 0.9999.

That is the whole of it. Saturn, Neptune and Pluto do not complete one circuit of the zodiac in fourteen and a half years; their longitudes never wrap even once in my file, and each tracks the calendar date at a correlation near 0.99. I did not hand my model the heavens. I handed it a clock, and it read the clock, and I applauded. Keep only the bodies that genuinely go round within my window — Sun, Moon, Mercury, Venus, Mars — and the sky predicts nothing at all: R² of 0.024, and −0.044 when I hold out the middle years. Mercury alone manages 0.011, and even that is the season, because Mercury never strays more than 27.3° from the Sun.

The RMSE was real. The comparison was not.

Which brings me to the sentence I added to this page in a flush of excitement: a crazy further reduction in RMSE was achieved by finally using a neural net. That reduction was real. It was also close to meaningless, and I can now say why with the file open.

In the deepnet post I trained on the first 80% of days and tested on the last 20%, and I reported that the error was dramatically smaller than methods based on the mean (average) rate of the training data or an approach assuming random chance. Look at what that comparison asks. My training period averages a rate of 0.293. My test period averages 0.187. The level of the series fell by more than a third between the two — most likely because browser spellcheck and phone autocorrect arrived and got good over exactly those years, which is a fact about software and not about Mercury.

Predicting the last 20% of days after training on the first 80% — my own split. R² scored against the training mean, exactly as I scored it in the deepnet post.

Method	R² on the test period
Predict the training mean — my baseline	0.000
Predict a constant: the last 27 days of training. No astronomy, no AI, nothing.	0.505
Predict a constant: the last 90 days of training	0.535
Predict a constant: the last 365 days of training	0.516

A single number, carried forward from the end of the training data — no planets, no network, no astronomy whatsoever — scores above 0.5 against the baseline I chose. Beating the mean of the training data on a series whose level has moved is not an achievement. It is the minimum any method clears the moment it notices the level has moved. I set my machine against a strawman and reported the walkover as evidence for astrology.

So the honest reading of my ridiculously good results is this: the deepnet noticed that misspelling rates had drifted down by the end of my training window, carried that knowledge forward, and beat an average I should never have been comparing against. To count as evidence it would have had to beat 0.505, not 0.000 — and nothing in that post asks it to.

Two loose ends I should tidy

The bracket above promises that a journal article, just approved for publication (March 2018), would be referenced in the bibliography. It links to a bibliography page that does not exist on this site, and eight years on the reference was never added. I have left the sentence standing, because a promise I did not keep is part of the record too.

And in the file itself, fifteen cells are not numbers at all. They read Mod[QuantityMagnitude[UnitConvert[Missing["RetrievalFailure"], "Degrees"]], 360] — Mathematica's ephemeris lookup failed on a few days for Uranus, Neptune and Pluto and wrote its own error message into my data where a longitude should be. I published that file for others to use, and asked them to tell me what they found. Nobody told me, and I never looked.

Where that leaves me. I reached for a neural net because the statistics were not giving me what I wanted, and the neural net gave it to me, and I did not ask why. The answer was there in the columns all along: I had fed it a calendar and called it a sky. There is a version of me that finds this funny — I spent three years and a conference talk proving, with increasingly expensive machinery, that Pluto knows what year it is. It does. So does a wall calendar, and the calendar is cheaper. The astronomy in this project was always real astronomy. It was just never doing the work I said it was.