On not editing encyclopedias any more

A correspondent writes with a suggestion for inclusion in the next edition of the Penguin Encyclopedia - and as messages of this kind are a fairly regular occurrence, this post provides an opportunity to tell interested parties of the main event this week, which means that there will be no more general encyclopedias under my editorship.

I have been editing general reference works for Penguin since 2002 - mainly The Penguin Encyclopedia and its Concise edition, as well as the Penguin Factfinder and its variants, such as The Penguin Book of Facts, and a set of small 'pocket' books on a range of topics. Earlier this year, Penguin decided not to do any further editions of these books. Why? Mainly because sales of published general encyclopedias have been falling away, thanks to the arrival of Wikipedia and other such sites. That is where most people go when they want information now, it seems - notwithstanding the fact that the information is regularly misleading, biased, or just plain wrong, as is inevitable when entries are compiled in a wiki-like way. Penguin isn't the only encylopedia publisher to withdraw. I read the other day of a French encyclopedia which has stopped publication for the same reason.

My editorial team in Holyhead continued to develop the reference database, nonetheless. Over the years we have been supplying reference data to a number of other publishers and online sources, such as biography.com, and the maintenance of the database continued to be a priority. A reference database is only valuable if it is up-to-date. We used to have a boast that if you died in the morning (and were famous enough) you would be in our database by the evening. The same coverage is needed for hundreds of areas of everyday life, from sports results to Oscars, from political leadership changes (in all countries) to the latest work by leading film stars and novelists. It kept my little team very busy, as you can imagine.

But you will have noticed the tense form, 'kept'. This week, the organization which owns the reference database, adpepper media, decided that its continued maintenance was no longer viable. My team was made redundant, and the database was immediately shelved. I therefore need to record that my intellectual responsibility for that database ceased as of this week. It's a great shame - for we are talking about a database which was compiled over a 22-year-period by hundreds of specialists, belonging to institutions as wide-ranging as NASA and the Natural History Museum - but in business terms it is perfectly understandable. If it doesn't make money, it's history.

Again for the record, let me summarize that history. The project began in 1986, when I was asked to compile the new Cambridge Encyclopedia for Cambridge University Press. Over the next decade, a series of reference books appeared from that publisher (the full details can be found in the books section of my website). In 1995, CUP changed its policy and sold the database to a Dutch IT firm, AND, who were more interested in developing the underlying taxonomy (which allowed us to mine the database for types of information) in relation to Internet search-engine activity (this was pre-Google, remember). The Cambridge encyclopedias continued to be published until 2000, but then AND went into liquidation, and all activity stopped, both on the reference side and on the IT side. Soon after, anxious not to waste the huge amount of work that had been done, and to safeguard the jobs of the Holyhead team (in an area which had been hard hit by recession), I and a colleague set up our own company, Crystal Reference Systems, to develop both the reference and the IT dimensions. We began to produce a new family of encyclopedias for Penguin, and to develop the IT side in a number of fresh directions - chiefly in relation to search engines, e-commerce, internet security, and automatic document classification. I will tell the story of that struggle on another occasion, but the upshot was that in 2006 the business was sold to adpepper media, who then shaped the IT side to focus on providing accurate and relevant placement of ads on web pages - what I call 'semantic targetting'. It is that side of the business which the new owners wish to concentrate on now, and the demise of the reference side of the business has been a consequence.

I hope to have the opportunity to tell the encyclopedia story at greater length in due course, and to acknowledge thereby the role played by the many specialists, editors, and readers who contributed to the projects over more than two decades. For the moment, this announcement must suffice to alert those reference-buffs who are aware of my involvement in general encyclopedia publishing that, as of 12 May 2008, that involvement is ended.