A recent post by Peter Turney lists the books that have influenced his research. As well as compiling a great list of books that are now on my mental “must read one day” list, he makes a crucial point for compiling such a list:
If a reader cannot point to some tangible outcome from reading a book, then the reader may be overestimating the personal impact of the book.
With that in mind I tried to think of which books had a substantial impact on my research career.
Although I can barely remember any of it now, the manual that came with the Commodore Vic 20 computer I read when I was around seven got me hooked on programming. In primary and secondary school it was that book and the subsequent Commodore 64 and Amiga manuals that set me on the road to studying computer science and maths.
In my second year at university I had the great fortune of being recommended Hofstadter’s “Gödel, Escher, Bach” by a fellow student. It is centrally responsible for getting me to start thinking about thinking and subsequently doing a PhD in machine learning. The fanciful but extremely well written detours into everything from genetics to Zen Buddhism also broadened my horizons immensely.
I. J. Good’s “The Estimation of Probabilities” was the tiny 1965 monograph I bought second-hand for $2 that made my thesis take a huge change in direction by giving it a Bayesian flavour. I now realise that a lot of that work had since been superseded by much more sophisticated Bayesian methods but sometimes finding a theory before it has been over-polished means that there is much more expository writing to aid intuition. It also helps that Good is a fabulous technical writer.
Philosophically, Nelson Goodman’s “Fact, Fiction and Forecast” also shaped my thinking about induction quite a lot. His ideas on the “virtuous circle” of basing current induction on the successes and failures of the past provided me with a philosophical basis for the transfer learning aspects of my research. I found his views a refreshing alternative to Popper’s (also personally influential) take on induction in “The Logic of Scientific Discovery”. Whereas Popper beautifully characterises the line between metaphysical and scientific theories, Goodman tries to give an explanation of how we might practically come up with new theories in the first place given that there will be, in general, countless that adequately fit the available data. In a nutshell, his theory of “entrenchment” says that we accrete a network of terms by induction and use these terms as features for future induction depending on how successful they were when used in past inductive leaps. This is a view of induction inline with Hume’s “habits of the mind” and one I find quite satisfying.
While not directly related to machine learning or computer science, there are a few other books that helped me form opinions on the process of research in general. I read Scott’s “Write to the Point” over a decade ago now but it still makes me stop, look at my writing and simplify it. My attitude to presenting technical ideas was also greatly influenced by reading Feynman’s “QED” lectures. They are a perfect example of communicating extremely deep and difficult ideas to a non-technical audience without condescension and misrepresentation. Finally, I read Kennedy’s “Academic Duty” just as I started my current post-doc and found it immensely insightful. I plan to reread it as I (hopefully) hit various milestone’s in my academic career.
Of course, like Peter, there are innumerable other books, papers and web pages that have shaped my thinking but the ones above are the ones that leap to mind when I think about how my research interests have developed over time.
Mark Reid May 26, 2008 Canberra, Australia