Gareth Rees

Blood

gareth.rees@pobox.com — Thu, 03 Dec 2009 21:31:50 GMT

I went to give blood this evening at the Wesley Methodist Church. The first thing I saw when I entered the hall was a man sitting at a table with his arm completely covered in blood. Not sure what had happened: maybe there was a problem with the collection equipment, a broken tube or dodgy clamp. But what an impression it would make if one were a first-time donor!

Cycling miscellany

gareth.rees@pobox.com — Tue, 01 Dec 2009 19:45:17 GMT

It's now more than a year since I was run down by a bus. Since then:

Response from Stagecoach: nothing
Response from Stagecoach's insurers: nothing of any use
Police response: no further action
Miles cycled: more than 4,000

So I think it's a win for me overall.

Fancy a bike ride on Sunday afternoon? I'm leading the CTC afternoon ride on 2009-12-06. Meet at 14:15 at the junction of Brookside and Lensfield Road.

The Guardian is normally free of the usual “Cyclists: Threat or Menace?” nonsense, but today it publishes an article by Edmund King, “Beware the iPod zombie cyclist”.

I wouldn’t normally consider this kind of prejudicial rubbish worth commenting on, but someone I respect appears to have fallen for it and there’s aren’t enough characters in Twitter to have a proper argument about it.

So, the first question is, who is Edmund King? Is he a sensible, neutral, commentator whose opinion on whether cyclists are “lycra louts”, “mindless maniacs” or “iPod zombies” is one we ought to take seriously? No, he’s the president of the Automobile Association, that’s who he is. He has a history of writing pro-motorist articles for newspapers, appealing for speed limits not to be reduced (for motorists), for motor vehicles not to be fitted with speed regulators, for Vehicle Excise Duty not to be increased, and so on. He’s a propagandist for the motor car, and no sensible person should read anything he says on the subject without checking their pockets afterwards.

So does what he says make any sense? No it does not. He sets up the trap with two sensible points: (1) you need to know if there’s another vehicle coming up behind you before you move laterally; (2) you need to concentrate while using the roads. These are true for cyclists as well as for motorists. He then slips in the falsehood: (3) cyclists need to hear vehicles behind them. This is wrong because some vehicles are silent (other cyclists) or too quiet to hear against the background noise of traffic (electric cars, or just quiet cars). You can't rely on your hearing: you have to look.

The stupidity of the whole piece is obvious when you consider that motorists, cocooned in their airtight cars, can hear very little at the best of times, and many motorists are listening to their own iPods via their much louder in-car entertainment systems. If it’s fine for motorists to cut themselves off from outside sounds, then why pick on cyclists? Conversely, if it’s bad for cyclists to do so, how much worse for motorists?

The rest of the article is the usual anti-cycling propaganda. “Cyclists keep getting killed by motor vehicles, so we must crack down on cycling.” (I paraphrase slightly.) Is it hard to imagine why the president of the Automobile Association might be keen to place the blame for cyclist fatalities anywhere but on the motorists he represents?

Update: in case you doubt my claim above that this Guardian piece is part of a propaganda campaign to blame cyclists for their own deaths, take a look at how the same story is reported in the Mail, where the equation is made explicit:

The fashion for wearing iPods while cycling has been blamed for a rise in the number of riders being killed or seriously injured. Dubbed the iPod zombies, cyclists who are distracted by thumping tunes blaring in their ears have become the latest menace on Britain's roads. Road safety campaigners fear the fashion for cyclists to wear earphones is partly responsible for the recent upsurge in injuries and deaths. Edmund King, the president of the AA, called for the Department for Transport to launch a campaign warning cyclists of the risk.

Much the same article appears in the Sunday Times.

It should be clear from these articles what's going on. The Department for Transport has just released its Transport Statistics Bulletin for April–June 2009. In this report one of the most salient figures is that 820 cyclists were killed or seriously injured in this period, an increase of 19% on the same period in 2008.

This figure obviously provides ammunition for pro-cycling campaigners in all their on-going battles for better facilities and changes to legislation. So you can see how important it is for the AA and other anti-cycling campaigners to get their spin in as quickly as possible. And in this case, the spin is that cyclists are to blame for the increase in casualties because they are "iPod zombies". Note that no figures are available for the number of "iPod zombies", or even any evidence that they exist at all. As the Sunday Times says, "It is not known how many of these [deaths and injuries] were caused by people listening to music because the DfT and the police do not record the information." In other words, maybe none of them. But that doesn't matter, because the prejudicial echo chamber is happy to repeat the spin. Meanwhile the president of the AA can pose as a "road safety campaigner".

The headline, which should be "Big increase in cyclist deaths and injuries", becomes "Beware, iPod zombie cyclists are on the rise".

Come on, people, show some media literacy!

Update 2009-12-12: I wrote this up in a more coherent form a "How the anti-cycling lobby poisons public discourse".

Why I hate FaceBook

gareth.rees@pobox.com — Sun, 25 Oct 2009 16:20:12 GMT

I’m sure FaceBook is fine for keeping track of what your friends are up to, but it seriously sucks for serious discussion. Seriously.

I can’t link to posts. Or rather, I can, but anyone who follows the link who isn’t logged in to FaceBook just gets “You must log in to see this page.” Why should someone have to set up a FaceBook account just to read my comment? And then when they log in they probably find they can’t read it anyway unless they are a “friend” of the poster.
I can’t find posts or comments via search engines.
Any work I put in to writing a comment is hidden away in FaceBook’s walled garden, where it just goes to waste. Why bother when only a set of people that the poster has marked as “friends” can read it?
There’s no syntax for putting links (or italic, or any kind of formatting) into comments. If I paste in a URL it might turn into a link, but then again it might not, depending on what characters are in the URL. And there’s no comment preview, so I can’t tell what’s going to happen.
Comment threads go into a stupid narrow column so that I can only see a couple of hundred words at a time.
Threads longer than two or three comments are hidden so that I can’t read them without clicking.
Individual comments longer than some random length, which seems to differ from one FaceBook page to another, get truncated so that I can’t read them without clicking.
This lack of space strongly encourages short comments, which in a serious discussion means you have to compress and abbreviate what you want to say, and that makes it likely that you'll be misinterpreted, as in the discussion on FaceBook that led to this long blog post on Robin Lustig. Of course I’d like to show you the original FaceBook discussion to back this up, but it would be pointless for me to do that, for reason 1 above.

So, you say, serious discussions aren’t what FaceBook is for? That would be fair enough if you didn’t start them there. I’d like to join in and contribute my opinions, but the whole experience of doing so annoys me so much that my contributions come out really cranky.

Obviously it's up to you what you write and where you write it, and who you allow to read it and comment, but I'd like to appeal to you to start public discussions somewhere public, and with a better commenting interface, at least if you would like me to comment on it without getting annoyed. (Of course you're free not to care about these issues, but I think that would be a shame.)

(This is a reply to a comment of chard on FaceBook which it would be pointless for me to link to for reason 1 above.)

Paying for postcodes

gareth.rees@pobox.com — Tue, 13 Oct 2009 18:56:13 GMT

David Howarth MP
House of Commons
London
SW1A 0AA

Dear David Howarth,

I would like to encourage you to sign Early Day Motion 2000, proposed by Tom Watson:

That this House notes with concern the attempts of Royal Mail to restrict access to the postcode database for not-for-profit web services; further notes with alarm that this heavy-handed approach has led to not-for-profit websites which seek to provide essential services to the public being left unable to function; and calls on the Government to ensure that the database is made freely available to anyone for not-for-profit use, thus enabling citizen-focused projects to flourish and innovate.

The Postcode Address File is an important piece of our national information infrastructure: it links postcodes with street addresses and geographical position. This data is critical for research, websites, and information services concerned with geographical locations within the UK. The Royal Mail owns this database, and charges users £3,750 per year ¹. This makes it wholly unaffordable for ordinary members of the public, community organizations, small charities, and so on.

Here are four examples of not-for-profit websites which would benefit from the Postcode Address File but cannot afford the current licensing fees:

PlanningAlerts sends e-mail when nearby planning applications are made. This is completely dependent on postcode information to determine which planning applications are where, and cannot be offered without some form of the Postcode Address File.
The Straight Choice collects election leaflets and other political material, and allows users to search for leaflets from their local area. This is also completely dependent on postcode information to determine which leaflets are local to where.
CycleStreets uses open-source mapping data to plan cycling routes. If it had access to the Postcode Address File, it could allow users to enter postcodes or street addresses as the endpoints of their route. At present, no such feature is possible.
Jobcentre ProPlus sends e-mail when local job vacancies are advertised. It cannot do this unless it can identify the location of the advertised jobs.

The ongoing expense of using the Postcode Address File means that there are many community websites and other applications that people would like to build, but cannot afford. We would all benefit if there were some way for individuals and non-profit organizations to freely license the Postcode Address File.

Yours sincerely,

Gareth Rees

¹ I’m not 100% certain about this figure: I’m presuming that an online service would need to have a “system licence” (covering a single computer, presumably a server responding to postcode lookup queries) or an “end user per click licence” (using the Royal Mail’s own online lookup API). But maybe there’s a cheaper option that I didn’t spot?

Background

From 2004 to 2006, the Postal Services Commission (Postcomm) held a public consultation into the future mananagement of the PAF. This was contemporary with the emergence of not-for-profit web-based public services (for example, the first MySociety web site, WriteToThem, was launched in 2005), so it’s not surprising that the public responses don’t make the kind of public benefit argument that I made above. Generally the concerns are about licensing terms rather than price, from the point of view of business users of the PAF, and especially from resellers and competitors.

Nonetheless, several of the public responses to the consultation argued for cheaper or free licensing of the PAF on public policy grounds. David Dorricott of AFD Software (a reseller of the PAF, and involved in a legal dispute over the terms of their license):

AFD’s view is that postcodes are an integral part of UK address lists and are ubiquitous in areas that are nothing to do with delivery—such as insurance underwriting or public administration. Database owners must be feed from Royal Mail’s continued claims to own property in their databases.

Christopher Roper of PointX (also involved in a dispute over PAF licensing) presented more detailed examples of public disbenefits arising from the high cost of PAF licences:

Is it reasonable, for example, for Royal Mail to maximise the revenue it derives from value added resellers and end users, even if such a pricing regime excludes legitimate use by, say, parts of the public and voluntary sectors and low volume commercial users? A major source of inaccuracy in the public registers maintained by the Environment Agency, for example, is the failure to use software that would ensure the transcription of PAF-compliant addresses. This may be a product of poor clerical procedures within the Agency, but it might equally stem from the requirement to pay for multiple PAF licences across all the Agency’s regional and area offices.

It is also the case that very few government agencies update postcodes on addresses they already hold as they are changed in PAF. For example the NHS</a> code service was, relatively recently, distributing addresses for all doctors in the country, at least 10% of which did not hold a currently valid postcode. This is a serious situation for suppliers of facilities data such as lists of doctors or dentists, but is even more serious where postcodes are used to re-code administrative data, such as the receipt of benefits, into statistical data sets for the ONS Neighbourhood Statistics Service. Insufficient research has been carried out to estimate the extent to which such statistics are compromised by inaccurate, or out of date postcoding; however it appears likely that the level of error in such data is high and that the expense and awkwardness of the present distribution mechanism for PAF is to blame.

Roper also noted some adverse consequences for his own business of the Royal Mail’s maximalist approach to intellectual property:

If, for example, one has a dataset containing a great deal of address data, it makes sense to check it against PAF to make sure the addresses are PAF compliant. However, if you then want to license your dataset to third parties, Royal Mail now claims that the third party has to have a valid PAF licence before they can use the dataset. A specific case where this arose concerned PointX [... which] built a Points of Interest database, designed for use by mobile telephone operators and others offering location based services over the internet. It provides exact map references for over two million addressable and non-addressable objects. For the addressable objects (business premises, churches, pubs etc) it provides a postal address.

The primary purpose of the data is not to provide postal addresses, but it seemed proper to check the addresses we were collecting for PAF compliance and to that end PointX had a PAF Licence. A small proportion of the addresses had missing or incorrect elements that were rectified from PAF. Because the PointX data is licensed on to third parties, we asked Royal Mail if our PAF license covered this use of PAF. We were told by Royal Mail that it did not, and that we would have to pay a royalty on our onward sales where our licensee did not hold a full PAF license, even though the proportion of PAF data in the PointX dataset might be less than 5%.

Keith Dugmore of the Demographics User Group suggested that the PAF should be freely licensed for the public good, though without detailed argument:

Members of the Demographics User Group view PAF as a vital data source for a range of purposes. The growth in its use throughout both commercial and government organisations in the last three decades has resulted in its becoming a de facto standard. We believe that this should continue, and be formalised with the PAF being recognised as part of the national data infrastructure. PAF should be maintained for the public good, and its use encouraged – to the benefit of Royal Mail’s operations – by being made free at the point of use, using OPSI’s Click-Use License.

Postcomm’s April 2007 decision noted the concerns but, as far as I can tell, offered no analysis of them and essentially reaffirmed the status quo, that the Royal Mail could set its own licensing terms and prices and if anyone didn’t like it, they could complain to Postcomm (see sections 6.48 and 6.49).

Charles Arthur criticized the Postcomm decision in the Guardian:

The Free Our Data campaign has a simpler solution: make PAF free online and remove the copyright restrictions on reselling it. The revenue impact on Royal Mail would be minimal; yet as well as engendering a lot of goodwill inside the public sector, doing so could encourage new private sector businesses presently put off by the cost. It's government-owned data; we should all get its benefit.

Skip a couple of years to 2009, and Harry Metcalfe was sufficiently annoyed with the Royal Mail that he created the website “Ernest Marples” (named after the former Postmaster General and fraudster). This site offered an API for converting postcode to latitude and longitude, which it allegedly accomplished by looking up the postcode in turn on one of the many websites that have paid a licence fee to Royal Mail and offer this service on their own website. (Google Maps is one well-known example of such a service, but I have no knowledge if Ernest Marples used Google Maps as one of its sources. Possibly the description of Ernest Marples’ algorithm was misleading and in fact it used a leaked copy of the PAF?)

Francis Davey pointed out that this novel attempt to (effectively) use the PAF without licensing it was likely to fail if legally challenged, because of the Royal Mail’s database right in the data.

Francis also noted that one of the points of Ernest Marples was to be sued:

As Harry [Metcalfe] explained at Open Tech 2009, part of the purpose of the site is political. There is a strong body of opinion that the Royal Mail should not have a monoploy on this extremely important database. If Ernest Marples is sued that will generate terrible publicity for the Royal Mail (as it should).

The Royal Mail duly obliged with legal threats against the site on 2009-10-02. This led to some publicity and Tom Watson’s Early Day Motion (though Julian Todd criticizes the lack of a more robust response, such as waiting to actually be sued).

Things you could easily do:

Ask your MP to sign Tom Watson’s Early Day Motion 2000.
Sign the a petition to the Prime Minister.
Vote for the PAF to released under a free license for non-commercial use by the Office of Public Sector Information.

I don’t want to pretend that free licensing for non-profits should be the ultimate goal here. It’s a change that would enable sites like PlanningAlerts to get up and running again, but it doesn’t really take on the general problem of part-public, part-private, datasets like the PAF. Just as it sucks to be a community website without the resources to license the PAF in the first place, it sucks to be a business completely dependent on the PAF for your own operations, but without the clout to negotiate sensible licensing terms.

Some kind of radical reform is needed, but we have to keep in mind that someone is going to have to pay for the maintenance of the PAF (a somewhat unbelievable £16 million a year, according to the Postcomm report). It wouldn’t be a victory for open source if competitors to the Royal Mail decided to develop their own post-coding systems. (Compare with the situation for mapping: many competitors to the Ordnance Survey prefer re-surveying the country to paying Ordnance Survey licensing fees.) I think it’s obviously a public good, so we’ll get the best value for money if taxpayers pay for it directly and license it freely to everyone, but good luck making that kind of case in the current economic climate.

Using html5lib to resolve relative URLs

gareth.rees@pobox.com — Fri, 09 Oct 2009 22:01:41 GMT

David suggested that the RSS feed for garethrees.org should have the full content of the posts instead of the summary. The way he put it was, “It should be up to me how to summarize your feed.”

I wasn’t so sure. At the time my own feed reader was the one built into Safari, which is adequate but not great, and my habit was to visit posts at their own URLs. But (also at David’s suggestion) I tried out Google Reader, and I can now see that with a decent feed reader it makes sense to read most posts in the reader.

I learned some other things from Google Reader. For example, I can see that there are 3 people who subscribe to my feed. Because the date that Google gives for a post isn’t the <pubDate> from the feed but is instead the date on which Google first discovered the post in the feed, I can tell that the first subscriber picked up the feed on 2009-08-12. Welcome, Google readers. You might also be interested in my shared items.

So, how to put the full posts in the feed? The main problem is that of translating relative URLs (in the post source) into absolute URLs (in the feed). It turns out that this has to be done for all relative URLs, even ones that consist only of a fragment identifier (that is, links within a single post), because Google Reader transforms <a href="#id"> into <a href="http://site-of-feed/#id">.

It turns out that this is really easy to do in Python using Google’s html5lib library.

import html5lib
import html5lib.serializer
import html5lib.treewalkers
import urlparse

# List of (ELEMENT, ATTRIBUTE) for HTML5 attributes which contain URLs.
# Based on the list at http://www.feedparser.org/docs/resolving-relative-links.html
url_attributes = [
    ('a', 'href'),
    ('applet', 'codebase'),
    ('area', 'href'),
    ('blockquote', 'cite'),
    ('body', 'background'),
    ('del', 'cite'),
    ('form', 'action'),
    ('frame', 'longdesc'),
    ('frame', 'src'),
    ('iframe', 'longdesc'),
    ('iframe', 'src'),
    ('head', 'profile'),
    ('img', 'longdesc'),
    ('img', 'src'),
    ('img', 'usemap'),
    ('input', 'src'),
    ('input', 'usemap'),
    ('ins', 'cite'),
    ('link', 'href'),
    ('object', 'classid'),
    ('object', 'codebase'),
    ('object', 'data'),
    ('object', 'usemap'),
    ('q', 'cite'),
    ('script', 'src')]

def absolutify(src, base_url):
    """absolutify(SRC, BASE_URL): Resolve relative URLs in SRC.
SRC is a string containing HTML. All URLs in SRC are resolved relative
to BASE_URL. Return the body of the result as HTML."""

    # Parse SRC as HTML.
    tree_builder = html5lib.treebuilders.getTreeBuilder('dom')
    parser = html5lib.html5parser.HTMLParser(tree = tree_builder)
    dom = parser.parse(src)

    # Handle <BASE> if any.
    head = dom.getElementsByTagName('head')[0]
    for b in head.getElementsByTagName('base'):
        u = b.getAttribute('href')
        if u:
            base_url = urlparse.urljoin(base_url, u)
            # HTML5 4.2.3 "if there are multiple base elements with href
            # attributes, all but the first are ignored."
            break

    # Change all relative URLs to absolute URLs by resolving them
    # relative to BASE_URL. Note that we need to do this even for URLs
    # that consist only of a fragment identifier, because Google Reader
    # changes href=#foo to href=http://site/#foo
    for tag, attr in url_attributes:
        for e in dom.getElementsByTagName(tag):
            u = e.getAttribute(attr)
            if u:
                e.setAttribute(attr, urlparse.urljoin(base_url, u))

    # Return the HTML5 serialization of the <BODY> of the result (we don't
    # want the <HEAD>: this breaks feed readers).
    body = dom.getElementsByTagName('body')[0]
    tree_walker = html5lib.treewalkers.getTreeWalker('dom')
    html_serializer = html5lib.serializer.htmlserializer.HTMLSerializer()
    return u''.join(html_serializer.serialize(tree_walker(body)))

Notes:

If you’re using html5lib for other tasks, you might want to apply my patch for a bug in html5lib.treewalkers so that the serialization works reliably. (For the application described above, I think the <body> won’t have any sibling elements, so you don’t need the patch.)
If your blog is in XHTML instead of HTML then your task is a bit harder. You can do the parse step easily enough using one of Python’s XML parsers, for example xml.sax. But then for the URL resolution step you have to take into account the XML Base specification, which is a bit hairier than the HTML <base> element. Though I suppose if you’re writing something for personal use only, and you don’t use xml:base yourself, you can get away with ignoring it.

Is Robin Lustig prejudiced against Japanese men?

gareth.rees@pobox.com — Thu, 08 Oct 2009 00:57:37 GMT

The provocative title of this post is intended to parallel the provocative title of Robin Lustig’s 2009-09-04 post to his blog on the BBC website, in which he asks, “Is Japan a dying nation?”

Mr Lustig introduces his post by explaining that he’s “on the plane back from Tokyo” after a week of reporting on the Japanese election. This could excuse quite a lot. Going to Japan for the first time is quite a mind-blowing experience for a Brit, especially for someone with interest in the Japanese nation: so many things are the same yet somehow different! or different but somehow familiar! You come back imagining that you have acquired a sophisticated understanding of Japanese society and you find yourself unguardedly retailing cultural generalizations. I’ve done all this myself, so I have a certain amount of sympathy. Nonetheless, were I to type up my naivety for everyone to see on one of the most-visited websites in the UK, then I think I’d deserve every bit of criticism that I’m about to heap on Mr Lustig’s article.

I have five criticisms of Lustig: (1) promoting offensive cultural stereotypes; (2) no links or citations to the evidence he relies on; (3) failing to give the context needed to understand his statistics; (4) promoting a bogus theory for Japan’s low fertility; (5) his general attitude to ageing and fertility.

1. Stereotypes

Here’s Lustig presenting an explanation for Japan’s low fertility rate:

I'm no social psychologist, so I wouldn't dare to come up with an explanation for why Japanese couples aren't having enough babies. But one theory is that Japanese women are increasingly reluctant to marry, because they think Japanese men have shown themselves unable to adapt to the needs of a new, more flexible society - and have retreated into a fantasy world of comics, video games and animated pornography where they feel less threatened. [... Japan] is the nation of manga comics and young women who dress up as French maids to pander to the fantasies of lonely men.

(For readers unfamiliar with modern Japanese culture: Lustig is referring here to otaku (obsessive fans, typically male), manga (comics), hentai (pornographic comics and animation), and cosplay restaurants.)

Note Lustig’s introductory caveats: it seems to me that he knows that what he’s going to say is offensive, and he doesn’t want to take the heat for it, so he takes care to distance himself from the explanation, palming it off as “one theory”. Whose theory is it, then?

Maybe if there were compelling evidence for this theory, then it would be fair for Lustig to mention it. But even were that the case (which it isn’t, as we’ll see later), he should have taken more care than this, just on the basic principle of politeness. This isn’t you and your mates at the Radio 4 studios, Mr Lustig. Japan is reading you and is not all that keen on what you have to say (see translations of the comments by Danny Choo).

It’s my opinion that Lustig picked this theory to write about because he finds it plausible and congenial to his prejudices, and because he knows that if he doesn’t give any of his sources, then it’ll probably be too much work for his readers to track them down and refute him.

2. References

This isn’t a particular fault of Robin Lustig, it’s a general failing of the whole profession of journalism. But it annoys me more each time I find myself spending time trying to track down the original sources behind some absurd claim in the media.

Mr Lustig, this is 2009. Close to 40% of 18-year-olds go to university, and many of them probably can’t remember the world without the web. Your readers are among the most highly educated people in the country. We know how to follow links and are as able as you are to read and understand original sources. So why persist in this absurd pretence that you’re the sole intermediary between scholars and a mass audience? You’re not fooling anyone any more. And maybe if you were conscientious about providing sources and references, then you’d have to think twice before parading your prejudices in public, which would be a win for everyone.

3. Context

Let’s take a few of Lustig’s statistics and see if I can find some sources for them and add some context.

First statistic:

A country that knows it is ageing more rapidly than any other major industrialised nation on earth.

The caveats “major” and “industrialised” suggest that Lustig suspects that this statistic isn’t as clear-cut as all that. And what does it mean to “age rapidly” anyway? Let’s take “speed of ageing” to refer to the change in the fraction of the population over 65 between 1999 and 2009. I don’t know for sure what Lustig means by a “major” nation or an “industrialised” one, but I’m going to assume that “major” means “population of 10 million or more” and “industrialised” means “GDP per capita (purchasing parity power) of $10,000 or more”.

This table gives the top ten major industrialised countries by speed of ageing (plus the UK), in roman text. Non-major or non-industrialised countries that would otherwise appear in the top ten are included but given in italics.

	% population 65 and over
Country	1999	2009	Difference
Japan	17	22.2	5.2
Virgin Islands	9	13.6	4.6
Georgia	12	16.4	4.4
Germany	16	20.3	4.3
Bermuda	10	14.2	4.2
Monaco	19	23.0	4.0
Albania	6	9.8	3.8
Korea, South	7	10.8	3.8
Korea, North	6	9.4	3.4
Saint Helena	8	11.4	3.4
Canada	12	15.2	3.2
Austria	15	18.0	3.0
Taiwan	8	10.7	2.7
Portugal	15	17.6	2.6
Greece	17	19.2	2.2
Italy	18	20.2	2.2
Mexico	4	6.2	2.2
Chile	7	9.1	2.1
United Kingdom	16	16.2	0.2

Data from the CIA World Factbook for 2009 and 1999.

So no qualifiers were in fact necessary, by the measure I chose. Maybe Lustig had a different measure in mind? If only he had said, or given his source, we might be able to tell.

Second statistic:

Which has the highest proportion in the world of people over the age of 65.

I think that when Lustig says “over the age of 65” he means “aged 65 and over” since the latter is a standard demographic category. The phrasing he uses is ambiguous between “65 and over” and “66 and over” which is why demographers avoid it. But this is an easy mistake for a non-specialist to make.

Using the data from the CIA World Factbook 2009, here are the top twelve countries by proportion of population 65 and over (plus the UK):

Country	% population 65 and over (est. 2009)
Monaco	23.0
Japan	22.2
Germany	20.3
Italy	20.2
Greece	19.2
Sweden	18.8
Spain	18.1
Austria	18.0
Bulgaria	17.7
Belgium	17.6
Portugal	17.6
Estonia	17.6
United Kingdom	16.2

Also from the CIA World Factbook, the bottom ten countries by total fertility rate (plus the UK).

Country	total fertility rate (est. 2009)
Singapore	1.09
Taiwan	1.14
Northern Mariana Islands	1.15
Japan	1.21
Korea, South	1.21
Lithuania	1.23
Montserrat	1.23
Belarus	1.24
Czech Republic	1.24
Bosnia and Herzegovina	1.25
United Kingdom	1.66

Third statistic:

on current trends, the population of Japan will have halved by the end of the century.

I’m pretty sure that Lustig is referring to the population projections made by the National Institute of Population and Social Security Research, which publishes 100-year projections on a five-yearly basis. The most recent is Kaneko et al. (2008), “Population Projections for Japan: 2006–2055”, Japanese Journal of Population 6:1. The title of the paper is a bit misleading because they include a long range (100-year) projection as well as a medium-range (50-year) projection.

Here’s figure A-1 from Kaneko et al., page 114, showing 50- and 100-year projections for nine scenarios using different predictions for fertility and mortality.

To get a bit more context for these projections, let’s compare with similar projections for Germany and the UK.

For Germany, I’ve chosen Eisenmenger et al., “Germany’s population by 2050: Results of the 11th coordinated population projection”, Federal Statistical Office.

This set of projections only goes to 2050, but the pattern up to that date is similar to Japan (a fall to 1950s levels by 2050). I didn’t find a longer projection, but I imagine that with suitable choice of fertility scenario, we could easily say, “on current trends, the population of Germany will have halved by the end of the century”.

For the UK, I’ve chosen Helen Bray (ed.) “National Population Projections: 2006–based”, Office for National Statistics, PP2:26, which projects the population to 2081. This is figure 9.6 from page 56, “Population of the United Kingdom according to principal and variant 2006-based projections, 1981–2081”.

Using this figure, we could write the opposite headline, “on current trends, the population of the UK will double by the end of the century!” But how much credence can we really put in these kinds of long-range projections? The problem is that population projection as it’s currently done is a process that iteratively applies fertility, mortality and migration rates to a population. This is a process that exponentially enlarges the differences between the scenario and reality. So you have to interpret long-range forecasts as saying, “if things go on like this,” while remembering that things won’t go on like that after all.

A salutary reminder of the difficulty of long-range population projection comes from Chris Shaw (2007), “Fifty years of United Kingdom national population projections: how accurate have they been?”, Population Trends 128, Office for National Statistics. His figure 1 on page 10 compares the actual population with six projections made between 1955 and 2004.

4. Causes

So, Lustig is accurate in his description of Japan as an ageing society. It’s a country that’s at the extreme on several demographic measures.

But it’s not a total outlier: Germany, Italy, and Greece are not far behind on proportion aged 65 and over. Germany in particular is (on current trends) looking at a similar population trajectory to Japan. Several countries have even lower fertility, and many European countries have total fertility rates in the range 1.2–1.4. This ought to give a careful writer pause: if several countries have low fertility rates and high (and increasing) proportions of people aged 65 and over, then we need to look to common explanatory factors before looking at cultural specifics.

Demographers refer to the change in population structure experienced by Japan as the “demographic transition”. It’s caused by falls in the rates of death and birth. The fall in death rates results from improvements in public health, hygiene, nutrition, and vaccination for childhood diseases. The fall in birth rates is due to improvements in the rights and social status of women, and the availability of cheap and effective contraception. Because the fall in birth rates happens later than the fall in death rates, there’s a period of rapid growth in population, which leaves a “bulge” in the population pyramid, an ageing society as this bulge reaches old age, and a declining population as members of the bulge eventually die.

However, since Japan is at the extreme end of the ageing continuum, there must be specific causes at work which serve to exaggerate the ageing trend. Maybe anime, otaku, and maid cafés do have a role to play here?

Here’s a plot of crude (per 1,000) birth and death rates for Japan from 1872 to 2008 (excepting 1944–1946, for which period there is no data). This could be a textbook figure for a demography primer.

Sources:

“Deaths and death rate by sex (1872–2004)”, Statistics Bureau of Japan.
“Live births by sex (1872–2004)”, Statistics Bureau of Japan.
Table 2.4 from Statistical Handbook of Japan, Statistics Bureau of Japan.

(The sudden drop in birth rates in 1966, and its immediate recovery in 1967, is striking. According to Robert William Hodge and Naohiro Ogawa, Fertility change in contemporary Japan, “In Japanese culture there is a traditional belief about the Year of the Fire Horse [丙午], which occurs once every 60 years. According to this long-standing superstition, a female born in a Year of the Fire Horse is destined to both an unhappy life and killing her husband if she marries ... many couples time their births so that they will not occur in such a year.” I’ll note in passing that unlike Lustig’s theory, this is an example of a specific cultural explanation for a change in fertility for which there is good evidence.)

The plot of birth rates shows two periods of sharp falls in fertility: the first from 1945–1960, the second from 1975–1990. It seems to me that if we’re going to look for specific cultural factors, we ought to find them in these periods, not in more recent fads like cosplay restaurants (2000s).

Hodge and Ogawa, in Fertility change in contemporary Japan, consider the distinctive features of Japan’s demographic transition from the perspective of 1991. From their foreword:

The present [i.e. 1991] Japanese situation is very similar to that of the West, where fertility rates in the range of 1.4 to 1.8 are common. All these countries, like Japan, share the experience of rapid population aging—an inevitable by-product of very low fertility. [...] Immigration, which offsets to some extent the negative natural increase in West Germany and many other Western countries, does not enter the picture in Japan, which has strong barriers against it.

Japan, like the Western countries, has achieved its very low fertility levels partly through later-age marriages but mainly through the increasing use of birth control measures—both contraception and abortion. [...]

While Japan resembles the West in its general demographic situation, there have been and are some important differences. Legal abortion has played a distinctively important part in Japan’s fertility decline, especially during the decades of the 1950s and 1960s. [...]

Japan is also unique in terms of the use of contraceptives. The condom is the most prevalent method, with rhythm in second place. [...] It is remarkable that the Japanese have maintained such low fertility for so many years despite the use of contraceptive methods that require exceptional care and discipline. [...]

Hodge and Ogawa go on to analyze other social factors: the “patriarchal extended-family system”, arranged marriages, the obligation to take care of parents in old age, education, urbanization, migration, and family size. None of the factors that Lustig mentions appear in the book.

For a slightly more recent look at this question, see Makoto Atoh (2001), “Very Low Fertility in Japan and Value Change Hypotheses”, Review of Population and Social Policy 10, pp. 1–21. Atoh considers the question of whether “secular individuation” (that is, lack of conformation to traditional religious values) is responsible for Japan’s low fertility. I suppose that by a very long stretch of concepts, you might consider this to be a proxy for the factors in Lustig’s theory. But this would be no help to Lustig in any case, because Atoh concludes:

According to various nationally representative time-series and comparable attitudinal surveys which have been undertaken in the post-war period by various institutes, there has hardly been any dramatic change in attitude toward religion and only a moderate change from social conformism toward individualistic attitude over the last 40 years. In contrast, there has been a tremendous attitudinal change related to women’s social and family roles, in such areas as premarital sex, divorce, gender-role division, and the care of elderly parents, especially since the middle of the 1980s. All these survey results suggest that the rapid rise in the proportion never married in Japan in this latest decade can be related to the change in the value system regarding women’s social and familial role and status, a change toward the valuation of a gender equal society, rather than to secular individuation or the end of a child-centered society.

An even more recent study is Naohiro Ogawa, Robert D. Retherford, Rikiya Matsukura (2006), “The Emergence of Very Low Fertility in Japan: Changing Mechanisms and Policy Responses”.

Most of the decline in the [total fertility rate] between 1973 and the present occurred because of later marriage and less marriage. (In this regard it should be noted that only about 2% of births occur out of wedlock in Japan.) [...] the main reasons for later marriage and less marriage after 1973 in Japan are the following:

Remarkable educational gains by women. The proportion of women of the relevant age enrolled into tertiary education increased from 5% in 1955 to 50% in 2005.
Massive increases in the proportion of women who work for pay outside the home. At present, about 99% of women work before marriage, and almost all of them are in paid employment, so that they have no financial reason for getting married.
A huge decline in the proportion of marriages that are arranged, i.e., from 63% in 1955 to 2% in 2002. Now, people have to find their own spouses, which is not so easy in Japan because the marriage market is not well-developed.
A major decline in the proportion of young couples who coreside with parents when they get married, i.e., from 64% in 1955 to 29% in 2002. Young couples increasingly do not want to live with parents, and the decline in coresidence makes it financially more difficult to get married and set up a household.
A major increase in premarital sex, thus implying that one does not have to get married to be in a sexual relationship. Between 1990 and 2004 the proportion of single women aged 20 and over who reported that they were currently using contraception rose from 39 to 57%.

To fully understand the baby bust since 1973, we should examine not only changes in the marriage component but also in the marital fertility component. To a considerable extent, marital fertility declined after 1973. [...] The main reasons for the decline [...] are:

the rise in direct costs of children (higher education is a major component of rising direct costs, which involve a substitution of quality for quantity of children),
the rise in opportunity costs of children for women (in terms of lost income as a result of temporarily dropping out of the labor force),
the shift in preferences away from children toward “other goods” (in other words, the “consumption utility” of children has fallen),
the fact that families are less secure,
the reality that women want more help from husbands in childrearing and housework

(I renumbered the reasons so I can refer to them uniquely.) The only one of these that seems to have any bearing on Lustig’s theory is reason (3): it might just about be the case that one of the obstacles to marriage in Japan is that young men are more interested in manga, anime, video games, etc. than they are in dating. But reason (4) seems to directly contradict this possibility.

In summary, I can find no evidence whatsoever for Lustig’s theory, and what evidence I can find seems to me to amount to a pretty compelling refutation.

5. Attitudes

In this section, I’m going to switch from facts to opinions, and consider some of Lustig’s implied attitudes.

First, the implication that ageing is primarily a problem:

According to one United Nations estimate, it'll need to import 17 million foreign workers over the next 40 years, just to keep its economy afloat and provide enough carers to look after the elderly. (By 2050, there will be more than a million Japanese over the age of 100.)

I wish that we had such problems in the UK: I’d love to have as good a chance of surviving to 100 as an average Japanese man of my age.

Second, the lack of any kind of appreciation that there are limits on growth, and ageing and population decline is something that every country in the world is going to have to learn to cope with if it wants to avoid environmental disaster. Japan’s extreme position on the continuum of ageing makes it a society that we could learn from, if we weren’t blinded by our prejudices.

Thirdly, the implication that couples somehow owe it to society to have “enough” babies. I disagree completely with this: if an increased birth rate is important to society, then society is going to have to pay for it. (Something that the Japanese government does in fact recognize, though £170 per month children’s allowance seems pretty feeble when set against factors 6–10 from Ogawa et al. that I quoted above.)

Lustig mentions the science-fiction film Children of Men as a dramatization of the effects of infertility. I’ll quote from Lois Bujold’s novel Ethan of Athos instead. On the remote planet of Athos, artificial uteruses permit the existence of an all-male society. Visiting the rest of the galaxy, Ethan learns about the mores of other worlds from Elli Quinn:

“Maybe they meant to raise battalions of mutant super-soldiers in vats like you Athosians and take over the universe or something”

“Not likely,” remarked Ethan. “Not battalions, anyway.”

“Why not? Why not clone as many as you want, once you’ve made the mold?”

“Oh, certainly, you could produce enormous quantities of infants—although it would take enormous resources to do so. Highly trained techs, as well as equipment and supplies. But don’t you see, that’s just the beginning. It’s nothing compared to what it takes to raise a child. Why, on Athos it absorbs most of the planet’s economic resources. Food of course—housing—education, clothing, medical care—it takes nearly all our efforts just to maintain population replacement, let alone to increase. No government could possibly afford to raise such a specialized, non-productive army.”

Elli Quinn quirked an eyebrow. “How odd. On other world, people seem to come in floods, and they’re not necessarily impoverished, either.”

Ethan, diverted, said, “Really? I don’t see how that can be. Why, the labor costs alone of bringing a child to maturity are astronomical. There must be something wrong with your accounting.”

Her eyes screwed up in an expression of sudden ironic insight. “Ah, but on other worlds the labor costs aren’t added in. They’re counted as free.”

6. Notes

Thanks to Alex Selby for badgering me into refining this argument, and for pointing me at Ogawa et al. (2006).

The fractal dimension of Mellon Udrigle

gareth.rees@pobox.com — Wed, 19 Aug 2009 22:06:36 GMT

Nicholas Crane (left) and Tony Mulholland (right) measuring a small section of the Scottish coast with a one-metre ruler.

Series 4, Episode 6, “Inner Hebrides to Faroe Islands”, of the BBC television series Coast, bravely tackled the fractal nature of coastlines. (The episode is available on BBC iPlayer until 2009-09-08 20:59 +0100. The segment I’m discussing here starts at 35:30.)

Visiting a stretch of the coast of Scotland near Mellon Udrigle, presenter Nicholas Crane and mathematician Tony Mulholland from the University of Strathclyde measured a short length of rocky coast using rulers of four different lengths, resulting in the increasing series of distances shown below right.

They namechecked Benoît Mandelbrot, whose 1967 paper “How Long Is the Coast of Britain? Statistical Self-Similarity and Fractional Dimension” connected the problem of measuring the length of a coastline or frontier with the mathematical notion of fractal dimension. But I think they missed an even more interesting story, because (as Mandelbrot noted) the coastline problem had been empirically investigated a little earlier by Lewis Fry Richardson.

Total length of this section of the coast, as measured with four different lengths of ruler. Note the inability of the programme makers to divide 123 by 2. Note also that this diagram bears little or no relation to the section of coastline actually measured in the programme: on Google Maps you can check that the orange line (labelled “14 m” here) is actually about 300 m long.

(I learned about Richardson’s story from Tom Körner’s book The Pleasures of Counting and what follows is a paraphrase of Körner’s account.)

Richardson was a Quaker and a pacifist. In World War I he served with the Friends’ Ambulance Unit attached to the French 16th Division, and transported wounded soldiers during the Third Battle of Champagne (1917). After the war he worked for the Meteorological Office on a system for forecasting weather by numerical computation, but when in 1920 the Met was placed under the control of the Air Ministry, Richardson resigned on principle.

While teaching physics at Westminster Training College, he carried out a mathematical study of the nature of warfare. His 1950 book Statistics of Deadly Quarrels systematically collects and analyzes data on wars and other conflicts between 1820 and 1945 to attempt to get quantitative answers to questions like “are some countries or groups inherently more belligerent than others?” “are wars getting more frequent or deadly over time?” “does a common language increase or reduce the chance that two countries or groups will fight?” and so on. He observed, for example, that occurrences of outbreaks of war between pairs of countries appear to obey Poisson statistics, suggesting that many events take place (provocations, accidents, disputes, assassinations etc) each of which has a very small chance of leading to a war.

The problem of the lengths of coastlines arose in the consideration of whether countries with longer borders are more likely to go to war. In order to normalise his data by the lengths of borders, Richardson needed to determine these lengths. However,

An embarrassing doubt arose as to whether actual frontiers were so intricate as to invalidate that otherwise promising theory. A special investigation was made to settle this question. [...] At first I tried to measure frontiers by rolling a wheel of 1.8 centimetres diameter on maps; but there is often fine detail, which the wheel cannot follow; some convention would be needed as to what detail should be ignored and what retained: considerable skill would be needed to guide the wheel in accordance with any such decision; and in practice the results were erratic.

Much more definite measurements have been made by walking a pair of dividers along a map of the frontier so as to count the number of equal sides of a polygon, the corners of which lie on the frontier. [...] Its total length, Σl, has been studied as a function of the length, l, of its side. This process comes down to us from Archimedes, and is standard in pure mathematics. [...]

The west coast of Britain from Land’s End to Duncansby Head was chosen as an example of a coast that looks more irregular than most other coasts in an atlas of the world. [...] As to how the total length Σl may be expected to vary with the length l of the side, I have no theory. Quite empirically the logarithms of these variables were plotted against one another; and a straight line was drawn through the points. More evidence would be needed before one could say whether the deviations from the straight lines are of any interest. I am inclined to regard them as random. The important feature for present purposes is that the slope of the graph is only moderate even for such a ragged line as the western shore of Great Britain. On the straight line in [the plot of log Σl against log l] the total length [Σl] varies inversely, as the fourth root of the side [l], that is

Σl ∝ l^−0.25

Richardson found that similar lines of fit could be drawn for other coastlines, with the situation summed up by “the useful empirical formula Σl ∝ l^−α where Σl is the total polygonal length, l the length of the side of the polygon, and α is a positive constant, characteristic of the frontier.”

Mandelbrot analyzed Richardson’s exponent −α as 1 − D, where D is the fractal dimension of the frontier. On Coast this concept was mentioned but the programme segment wasn’t long enough to explain it, even at the empirical level of Richardson’s analysis.

So let’s follow Richardson’s procedure ourselves for the Coast measurements, using gnuplot to do the fitting. Here’s the file coast.data:

0.5 61.5
1.0 51.0
2.0 30.0
14.0 14.0

(In the anomalous case of the 0.5 m ruler, I’ve taken the number of steps, 123, as being correct, based on Nicholas Crane being filmed clearly counting to 123. This gives a total length of 61.5 m, not the 64 m reported by the programme.)

Here’s the gnuplot program:

set logscale x
set logscale y
set xlabel "Length of ruler (m)"
set ylabel "Measured length of coast (m)"
f(x) = b * x**(-a)
fit f(x) 'coast.data' via a, b
plot 'coast.data' title "Measurements", f(x) title "Best fit"

The result is shown on the right, and the best-fit procedure finds the constant a ≅ 0.46, giving this very wiggly piece of coast the fractal dimension of 1.46.

Emacs 23

gareth.rees@pobox.com — Wed, 19 Aug 2009 15:28:17 GMT

The Emacs 23 icon, in SVG.

Emacs 23.1 was released on 2009-07-29. In this review I’ll describe some of the new features in Emacs 23 that seem particularly useful to me, and some features from older versions of Emacs that I’ve only discovered recently, or which I think will be interesting or novel to the Emacs users among my readers. But everyone has different requirements and preferences, so if you find something particularly useful that I’ve missed, why not add a comment? (But if all this just reminds you of how much you’ve always hated Emacs, I’d appreciate it if you could vent your spleen elsewhere, thanks.)

Installing

I had no trouble getting Emacs 23.1 working on Windows: the FSF provides native Windows distributions. Emacs has the easiest installation process of any Windows software I know of: you just unzip the distribution into C:\Program Files or wherever and you’re done. If only other Windows software was so easy to install.

On Mac OS 10.5.7 I tried building from source following the EmacsApp instructions (./configure --with-ns && make && make install), which failed. I tried again after upgrading to 10.5.8 and this time it worked. I have no idea what went wrong, and have no particular desire to go back to 10.5.7 and investigate. If you have problems building Emacs 23, you might try one of the universal binary distributions at Emacs for Mac OS X (which I think could be better advertised).

On Mac OS X I experienced a couple of configuration glitches. First, I found that the Alt key had been assigned to the “Alt” modifier flag, a feature which I don’t use (I prefer the native use of the Alt key for entering accented letters and other characters). I fixed this by customizing ns-alternate-modifier. Second, there’s a problem with the PATH environment variable: Emacs gets the system PATH, not the one from my .profile. This problem is noted on the Emacs wiki together with a solution, but if this is a general problem with Mac OS X as claimed there, why am I discovering this for the first time in Emacs 23?

Native Unicode support

From NEWS:

*** The Emacs character set is now a superset of Unicode.
(It has about four times the code space, which should be plenty).
The internal encoding used for buffers and strings is now Unicode-based and called ‘utf-8-emacs’

What I noticed was an immediate improvement in the handling of some large UTF-8-encoded files I’m working on that contain text from several languages. These now open instantly, whereas in Emacs 22 there were delays while Lisp code was run to decode the characters and convert them to Emacs’ internal encoding.

This change probably doesn’t seem like all that much of a big deal. “It’s 2009 and Emacs has only just switched to Unicode internally?” you might well ask, “What a bunch of late-comers!” But in fact Emacs was early to the party, not late: the difficulty in switching to Unicode internally is because Emacs has had multilingual character support since August 1993 (in the form of MULE 1.0), when Unicode was at revision 1.1 and arguments about Han unification were still a live issue. At this point you had to have very good foresight or a lot of luck to see that Unicode was going to be the solution to multilingual character encoding. (So all credit to the Windows NT developers.)

From the perspective of 1993, it’s not at all obvious that Unicode is a good idea. Out in the real world, people want to edit text in all sorts of character encodings, and the most natural way to support this is to remember which encoding you are working in, and represent the data natively in that encoding. That way you can be sure that you don’t break anything, and it’s easy to extend to support more encodings. In the Unicode paradigm, you have to translate your input from its encoding into Unicode, using large mapping tables defined for this purpose by the Unicode consortium, and then when you write the data back out again you apply the mapping tables in reverse. If you’re going to adopt this approach, you’re going to need a lot of confidence that the Unicode consortium to get these mapping tables right (in particular, the tables need to be injective: different code points in other encodings need to be mapped to different code points in Unicode). You also need the disk space and memory for the tables, which was more of an issue back then (when people used to criticize Emacs for being a bloated application; nowadays Emacs.app is about the same size as iCal.app).

So I don’t think we can criticize the MULE developers for failing to adopt Unicode, and in any case they were constrained by design decisions made in MULE’s predecessor Nemacs (“Nihongo Emacs”) which was first released in June 1987, long before the publication of Unicode 1.0.0 in October 1991.

However, the very success of MULE (in 1993 Emacs was streets ahead of other editors when it came to editing multilingual text, the main difficulty for the user being acquiring the necessary fonts) ended up leading Emacs down a bit of a blind alley. The difficulties of merging MULE into GNU Emacs (not the least of which were the copyright problems) lasted until 1997, when Emacs 20.1 had multilingual support built in. And it wasn’t all that long afterwards that it became clear to everyone that Unicode was the way to go.

So the MULE internals had to be backed out and replaced with new Unicode internals, without breaking too much Emacs Lisp code that interacted with characters and their encodings. This exacting project was carried out by Ken’ichi Handa of the AIST in Tokyo. Some measure of the complexity and delicacy of this ten-year process can be gathered by browsing Handa’s e-mails to the emacs-devel mailing list. The 2003 thread “eight-bit char handling in emacs-unicode” is a good example: Simon Josefsson reports a bug whose cause is that the Emacs Lisp implementation of RFC 2104 (message authentication using cryptographic hash functions) is using Lisp strings to represent arbitrary sequences of 8-bit values (what Emacs Lisp calls “unibyte strings”). This worked fine in MULE, but because of the way it constructs these strings, in the Unicode branch of Emacs these strings end up consisting of encoded characters instead (what Emacs Lisp calls “multibyte strings”) and the message authentication fails.

Improved font support

From NEWS:

** New font code.
Fonts are handled by new code capable of dealing with multiple font backends. This uses the freetype and fontconfig libraries.
*** Added support for local fonts (fonts installed on the machine where Emacs is running).
*** Added support for the Xft library for antialiasing.
*** Added support for the otf library for complex text layout by OpenType fonts.
*** Added support for the m17n library for text shaping.

I noticed that on Windows Emacs seems much better at finding fonts for all the characters it needs to display. For example, Emacs 22.3 on Windows was unable to find a font containing ～ (U+FF5E FULLWIDTH TILDE) even though I have several such fonts (resulting in a “missing glyph” box). Emacs 23.1 displays it fine.

The ucs-insert command completes on Unicode character names using wildcards.

Character entry by Unicode name

Suppose you want to enter a MULTIPLICATION SIGN but don’t know how to type it on your keyboard and you don’t happen to remember its Unicode character code. It used to be the case that you had to hunt around in the Character Palette (on Mac OS X) or the Character Map (on Windows). In Emacs 23.1 you can type C-x 8 RET MULTIPLICATION SIGN RET. (C-x 8 RET is the rather awkward key sequence for the ucs-insert command; I suppose I’ll get used to it eventually.)

Actually the situation is much better than that because there’s completion on the set of Unicode character names, so you can just type C-x 8 RET multip TAB s TAB RET. Another example: you can type C-x 8 RET greek capital TAB to get a list of Greek capital letters. And there’s wildcard completion too, so if you want some kind of arrow but you’re not sure which, you type C-x 8 RET *arrow TAB and get a list of all the arrows in Unicode. I think I will have a HEAVY BLACK-FEATHERED NORTH EAST ARROW, please: ➹.

The describe-char command now shows the Unicode name of the character, and a (customizable) set of character code properties, for example:

Character code properties: customize what to show name: HEAVY BLACK-FEATHERED NORTH EAST ARROW general-category: So (Symbol, Other) canonical-combining-class: 0 (Spacing, split, enclosing, reordrant, and Tibetan subjoined) bidi-class: ON (Other Neutrals)

The list-charset-chars command is also useful: M-x list-charset-chars RET unicode-bmp RET gives you a buffer containing the entire Unicode Base Multilingual Plane.

Native word wrapping

Some history. Word wrapping has previously been a bit of a pain in Emacs. It used to be that you were expected to wrap (“fill” in Emacs terminology) all paragraphs by hand using fill-paragraph (M-q). This was obviously a waste of your time, so you could turn on auto-fill-mode to automatically break lines that got too long. But this only works at the end of a paragraph: if you’re editing lines in the middle of a paragraph, auto-fill-mode doesn’t refill the paragraph for you. (There were add-on libraries for doing this, like Per Abrahamsen’s maniac.el—tagline: “fill paragraphs like a maniac”—and Dave Love’s refill.el.)

However, the problem with all these approaches is that when you “fill” a paragraph, you lose information about which line breaks were inserted deliberately by the author (and should be left alone), and which were inserted automatically for the purposes of wrapping to a certain width (and which can be adjusted automatically). For some kinds of editing, you need to preserve this information.

Emacs 22.1 provided longlines-mode, which distinguished between these two types of line (so-called “soft” and “hard” newlines) using text properties. But this had its own problem, because the soft newlines were inserted when a file was opened, and then removed just before the file was saved, meaning that line numbers were inconsistent between the file on disk and the corresponding buffer in Emacs. And this meant that if you wanted to visit a line in the buffer based on a line number in the output of a command-line tool like grep or p4 diff, you were out of luck.

Emacs 23.1 finally gets it right by adding word wrapping to the display engine, under the control of the visual-line-mode command.

A use-case for transparent editing windows: composing an e-mail while looking at a map, on a screen that’s so small that you can’t position the windows side-by-side. (See also the full-size version.)

Frame transparency

You can change the opacity of your frames by setting the alpha frame parameter to an integer from 0 to 100 (meaning percent opacity). There’s no user interface for this yet (as far as I can tell), but you can evaluate Lisp code like (modify-frame-parameters nil '((alpha 75))).

I used to think that frame transparency was just an amusing thing you could do to show off the rendering capabilities of your windowing system. But on my 12-inch Powerbook, I have found myself really wanting this feature, so that I can simultaneously look at a full-screen document (or video, image, diagram, etc) and compose text in Emacs. On a bigger display I could position the windows side-by-side, but on my laptop there’s not enough room to do that.

Incremental search

When incremental search fails, the search string you entered is coloured to show the initial portion that matched and the remainder that didn’t match. This makes it quicker to recover from typos.

Line numbering

At long last, a proper line numbering feature (with the line numbers in the editor margins, rather than inserted into the buffer): linum-mode.

File completion with wildcards

(New in Emacs 20) You can type C-x C-f *.c TAB and get a list of all your C files to select from. (This probably works best if you’re using a mouse so you can click on the completion you want; typing C-u - C-x o to move to the completion window gets a bit tedious.) You can also type C-x C-f *.c RET to open all your C files in separate buffers.

Edit file names interactively

(New in Emacs 22) Suppose you have a whole bunch of files that need to be renamed: perhaps their files extensions need to be consistently lowercase, or you’d like to suffix each one with the year it was created. This has always been a bit of a tricky problem: you can roll your own solution in shell script, but it always ends up being hairier than you’d like:

for EXT in JPG PNG; do
    for FILE in *.$EXT; do
        mv $FILE $(basename $FILE $EXT)$(echo $EXT | tr A-Z a-z);
    done;
done

Or you can use a specialized tool like rename (if you have it). Perhaps something like this:

rename -v 's/\.(JPG|PNG)$/.\L\1/' *.{PNG,JPG}

(That’s untested! I hope you remember to use the -n option whenever you use rename.)

Anyway, in Emacs 23 you can visit a directory in Dired mode (C-x d) and then run the dired-toggle-read-only command (C-x C-q) and then edit the filenames by typing, by using query-replace-regexp or string-rectangle, or however you like. When you’re done, type C-x C-q again and each file whose name you changed gets renamed.

This seems like magic the first time you do it, but it’s really handy not just because you have all the editing features of Emacs at your disposal, but because you get a preview of your changes: you can look over them, correct mistakes, and revert them if you don’t like them (using the revert-buffer command as usual).

(You can also edit some of the other file metadata in the Dired buffer, such as the permissions.)

Replacement with evaluation

Emacs 21 introduced the commands (query-)replace-regexp-eval, which are like (query-)replace-regexp but instead of a replacement string, you provide an arbitrary Lisp expression to generate the replacement text. Emacs 22 obsoleted these functions with three new escape sequences for the replacement string in (query-)replace-regexp:

\? is a placeholder for a string that you enter interactively for each replacement. (It only works for replace-regexp, not query-replace-regexp, where you can always type e to edit the replacement text. For example, you can add comments to all your uncommented integer variables by typing M-x replace-regexp RET ^\s-*int\s-+\s_+;$ RET \& /* \? */ RET.

\# expands to the count of replacements already made by the replacement command. So you can number the lines in your file (starting at zero) by typing C-M-% ^ RET \# SPC RET !. Not all that useful by itself, but very useful in combination with:

\, evaluates the following Lisp expression, converts the result to a string and includes it in the replacement text.) Back-references like \1 can appear in the Lisp expression (where they get turned into (match-string 1) etc.), as can escape sequences like \# and \?. For example, to number the lines in your file starting at 1, type C-M-% ^ RET \,(1+ \#) SPC RET !. To multiply all the integers in your file by 5, type C-M-% [0-9]+ RET \,(* (string-to-number \&) 5) RET !. (A slight shame about the need for string-to-number there.)

\, is a feature that I use almost every day.

Emacs Lisp as a scripting language

(New in Emacs 22) You can put #!/usr/bin/emacs --script at the top of your Emacs Lisp files and run them from the shell.

Some cycling observations

gareth.rees@pobox.com — Sun, 19 Jul 2009 22:06:21 GMT

When I was cycling to Warsash the other week, I stopped at the George in Odiham and ordered their sticky toffee pudding. It was very large and very sweet, and even though I couldn't finish the whole thing, afterwards I wasn't hungry for more than 30 rather hilly miles, which suggests that the whole pudding had a lot more than 1,000 kcal.

It was just what I needed, but the hotel can't get all that many long-distance cyclists or manual labourers. So who is it for?

Riding to Stradishall with the Cambridge Cycling Club.

One of the things about the Tour de France that may seem odd to non-cyclists is that there are three separate competitions going on at the same time: the general classification, based on total overall time (yellow jersey); the points competition, for winning sprints and stages (green jersey); and the king of the mountains competition, for winning climbs to the tops of mountain passes (polka dot jersey).

Why is this? It’s because people with different body types are good at different types of cycling, and the multiple competitions allow more cyclists to take part competitively in the race. Cyclists with large bodies and a preponderance of Type I (“fast-twitch”) muscle fibres are good sprinters; cyclists with small bodies and a preponderance of Type II (“slow-twitch”) muscle fibres are good climbers; and cyclists with intermediate physiques have a chance at the general classification.

The difference between these types of physique is quite dramatic. I’m a climber, so when I go out with the Cambridge Cycling Club, I can keep up with cyclists who are substantially faster than me on the flat, so long as there are enough hills on our route. On the climbs, I use my better power-to-weight ratio; on the descents and flats, they use their better power-to-surface-area ratio to leave me behind.

ActiGraph GT3X activity monitor.

I'm taking part in a study on commuting and health in Cambridge run by the MRC Epidemiology Unit at the Institute of Metabolic Science at Addenbrooke's.

My participation in the study involves wearing an ActiGraph GT3X activity monitor for a week to see how much exercise I get. (In passing, $335 seems like a lot of money for a device which basically consists of a three-axis solid state accelerometer and 4 MB of memory on a 3 cm PCB. I guess that’s low production volumes for you, and maybe the rigours of medical certification: a Wii Remote is a much more sophisticated piece of kit, and that retails for $40.)

It will be interesting to see (when or if the researchers eventually publish) how they’ll deal with two obvious sources of bias. First, their sample is (at least partly) self-selected (they recruit participants who sign up at their website; I don’t know what other avenues of recruitment they have), and these participants will differ from the general population in lots of ways, not all of which will be easy to control for. It seems quite likely that people who volunteer for a study on activity will be more active than the population in general. Second, the problem of reactivity: people change their behaviour when they are being observed. It seems certain that someone whose activity is being monitored will undertake more activity than they normally would. For example, if I weren't taking part in the study I might have taken more notice of the weather forecast and not done any cycling this afternoon. However, this second source of bias applies to all participants in this study (and indeed in any study that monitors activity), so it probably doesn't invalidate comparisons; it's just the absolute numbers that are affected.

Azara Blog points out a third source of bias:

What is the point of this survey? The obvious conclusion will be that people who cycle or the half dozen people who walk to work are healthier than those who drive or take a bus. What a surprise. The eventual report will probably miss the fact that there is a difference between correlation and causation. So for example, for people who work in Cambridge, richer people generally live closer to their workplace and are more likely to cycle, and of course richer people are generally healthier.

The particular causative relation alleged here (richer people choose to live closer to their workplace) may or may not be true, but it illustrates a potential source of bias: commuting distance is not an independent variable because peoples choices about where to live and work may be correlated with their choice of commuting mode.

However, the blogger goes on to suggest that the researchers may be motivated to ignore this source of bias in order to reach politically correct conclusions:

But the way the conclusion of the study will be pushed is that if only the peasants could be forced to cycle 10 miles, or walk 5 miles across a muddy field, to get to work, then the world would be a better place. It is unfortunate that at a time when the UK research councils are going to be forced to tighten their belts, that money is diverted from real research to this kind of pointless academic middle class exercise.

Advanced Photo System

gareth.rees@pobox.com — Thu, 16 Jul 2009 10:51:05 GMT

Advanced Photo System logo.

Advanced Photo System was a 24 mm film format introduced in 1996 by a consortium of film and camera manufacturers, including Kodak, Fuji, Canon, Minolta and Nikon. The Canon Elph (Ixus in Europe) was released the same year. It was a beautiful camera: according to Wikipedia “at the time, the world’s smallest autofocus camera”.

Elph 370Z, a compact autofocus APS camera released by Canon in 1998. Photo by pointnshoot. Licence: CC-BY.

My camera was an Ixus Z70, the European version of the Elph 370Z you can see at the right. The bulging lens housing marred the elegant lines of the original Elph/Ixus, but it did have the compensation of 3× zoom. I used it for about five years and shot 34 rolls: 1,080 photos (which may not seem like very many in this digital age, but the fact that each exposure cost about £0.40 was a bit of a restraint). The camera was stolen when my house was burgled in 2005.

1,080 photos.

APS was a very short-lived film format. It arrived at the wrong time, just as film was about to be replaced by digital in almost all niches of the photography business. By 2000, digital cameras were clearly a better choice in the ultra-compact autofocus niche where APS had been most successful, and in 2004, only eight years afer introducing the format, Kodak stopped manufacturing APS film altogether. Ever since I heard the news, it’s been bugging me that I ought to get round to digitizing my APS photos while it’s still possible to find someone to do it. Minority formats have a habit of becoming obsolete quickly, and if I wait too long I might find that it’s no longer possible to convert the format. So from time to time I’ve been looking around for someone who can offer me a good price for digitizing all my APS film.

The curious thing about this search is that several photography businesses have felt obliged to explain to me why I shouldn’t have been using the APS format in the first place. A salesman at Campkins Cameras on Rose Crescent spent a couple of minutes lecturing me on how the APS format had been forced on the photographic industry by the manufacturers and that they hadn’t wanted to adopt it at all (Campkins can’t digitize APS film, in case you were wondering). Another company responded to my query with the admonishment, “I'm sorry but we do not have any facilities to either print or scan APS film as it considered an amateur film and we mainly deal with professionals”. Yes, thank you, photography companies, I don’t need you to tell me that I’m a rubbish photographer, I only wanted to find out if I could, like, pay you to digitize my photos.

I guess this is a case of professionals not understanding that amateurs can have very different requirements. Here’s an article about APS by Philip Greenspun from 1997 showing the same slightly blinkered attitude: “An APS negative is 56% the area of a 35 mm negative. That's all that a serious photographer really needs to know about the format. Everything else is gadgetry.” However, Greenspun includes a very sensible counterpoint from Kleanthes Koniaris, and a lot of the comments are insightful too. Some of them are quite funny, like this one from 1999: “Like digital photography, I feel that APS will have little or no effect on professional photography.” How’s that prediction looking now?

The key advantage of APS for an amateur like me was that it enabled manufacturers to make cameras small enough to slip into a pocket and robust enough to carry around in that pocket all day, and that made it much more likely that the camera would actually be to hand when I wanted to take a photo. It’s no good owning some fancy-pants piece of kit if its size and delicacy means that you don’t carry it with you. There’s no way I would have wanted to carry a 35 mm camera up a cliff or down a couloir, but the Ixus went with me in my trouser pocket. A poor quality photo is a lot better than no photo at all, and anyway, how likely is it that I’d get a good quality photo even if I did have a fancy camera? I know from my attempts to take photos with my parents’ Leica M4 that I don’t have the patience for anything beyond point-and-shoot.

APS also made some user interface improvements over 35 mm: there was no difficulty in spooling the film (you just dropped the cartridge in via a hatch); no chance of double exposures or missed frames (all winding was automatic); no way to ruin exposures by accidentally opening the back (the hatch locked until the film was rewound into the cartridge); no confusion between shot and unshot rolls (the cartridges had an indicator, and anyway the camera refused to shoot film twice). This kind of thing doesn’t matter to the professional who has long since refined their working methods to the point where they forget how novice mistakes are even possible.

Anyway, there are plenty of companies out there who will digitize APS photos, but most are a bit expensive for someone with 1,080 exposures. Jessops on Green Street wanted to charge me £0.50 per exposure, about twice the cost of developing the film in the first place. I was about to write, “at that price it would have been cheaper to buy my own scanner,” but I’m not sure that’s true: no-one seems to make APS batch scanners (like the CanoScan FS2710 or the Minolta Scan Dual IV) any more, so buying one would involve a lot of hunting around on auction sites, with no guarantee of finding a working model at a reasonable price.

Eventually I found PictureLizard in Swindon, which has very generous volume discounts: I paid just £0.15 an exposure. The quality is, to be honest, not that great: the scans are about 1,600 dots per inch (about 1,500 pixels across a 24 mm exposure) and are rather grainy (on the other hand, the originals were not all that great either). One of the rolls was left-right reflected; it must have been fed into the scanner back to front. The photo metadata (dates and times) were not captured by the scanning process. The accidental reflections were easy to fix; restoring the dates (by reference to the printed copies) took some time. Still, at that price I can’t complain.

Here are twelve of my favourites. Click on the thumbnails for larger versions.

Finish of the London to Brighton charity bicycle ride, 1998-06-21.	Pavey Ark and Stickle Tarn from Harrison Stickle, English Lake District, 1998-07-18.	drj11 climbing Demo Route (HS), Sennen, Cornwall, 1998-08-14.
Verbier, Switzerland, 1999-02-12.	Martin Moran climbing March Hare’s Gully (IV), Beinn Bhan, Scotland, 1999-02-24.	chard climbing Southwest Corner (5.8), Eldorado Canyon, Colorado, 2000-08-12.
Upper Langtang Valley, Nepal, 2001-04-05.	River Cam bursts its banks and floods Stourbridge Common, Cambridge, 2001-10-23.	Loch Quoich and Killilan from the summit of Maol Chinn-dearg in Glen Shiel, Scotland, 2003-06-06.
Looking down March Hare’s Gully on Beinn Bhan at an ice-covered lochan in Coire na Poite, 1999-02-24	drj11 climbing Lancet Crack (VS), Brimham Rocks, Yorkshire, 2000-07-16.	writinghawk on a bus in Nepal, 2001-03-30.

Cycle tour to Warsash and Staines

gareth.rees@pobox.com — Tue, 30 Jun 2009 19:00:06 GMT

At the weekend I went on a cycle tour to Warsash and back:

Friday: Cambridge to Warsash, 220 km (137 miles)
Sunday: Warsash to Staines, 107 km (67 miles)
Monday: Staines to Cambridge, 131 km (81 miles)

Click the photos for larger versions.

Key to photos (anticlockwise from top left):

A foggy morning on the A603 in Cambridgeshire. It was cool and damp at 05:30 and a light fog blanketed the fields.
The Greyhound pub on the B656 near St Ippollytts in Hertfordshire. It was open at 07:30 on Friday for breakfast, and with a sign offering a “cyclist special”, how could I resist? Coffee and waffles were just what I needed. (I took the photo on the way back, which is why it is sunny. Sadly the pub was closed at 14:30 when I passed on Monday, so I can’t comment on its lunches.)
Eversley Cross in Hampshire with a view of the downs.
Scrubbs Lane near Bishop’s Sutton in Hampshire, with the prospect of yet another hill.
An awkward junction in the South Downs, Hampshire. On Sunday afternoon I got quite lost in this little maze of lanes southwest of Empshott. The sign points to Froxfield, Petersfield, and Privett. I did not come from any of these places, nor was I going to any of these places. And the fourth arm of the crossroads was missing from my map, but very present on the ground. I ended up going the correct way, but I was quite certain I had gone wrong.
Another awkward junction in the same maze of lanes. The sign points to Froxfield, Priors Dean, and Hawkley. I wasn’t going to any of these places either. This area is very beautiful and would reward exploring with a better map or more accurate directions!
Empshott Green, Hampshire. At last, I had escaped from the maze.
A lake by Thursley Road a few kilometres south of Elstead in Surrey.
Puttenham Church, Surrey.
Crossing the Thames in Staines on the A308.
Heathrow Airport from the A3044.
Whitwell, Hertfordshire.
Wimpole Hall from the A603 in Cambridgeshire.
The first view of Cambridge from the A603. A big relief: from here there’s less than 15 kilometres to go.

Colin Bell asked, “Do you have any good resources for planning long-distance rides? Some friends are wanting to do Cambridge–Chepstow...” So here’s more than you might ever want to know about how I planned the trip. Bear in mind that this is the first multi-day cycle tour I’ve done, so I’m probably not a good fount of knowledge here. For advice from experienced cycle tourists, you probably want to try the Touring & Expedition section of the CTC forums.

I planned the trip using the “Get Directions” feature of Google Maps. With mode set to “Walking”, it does a pretty good job of generating cycling routes. (Perhaps that’s because it doesn’t yet know about very many footpaths; when it does, the “Walking” directions will be less useful to cyclists ... but maybe Google can add a “Cycling” mode which avoids trunk roads as well as footpaths. If any Googlers are reading this, please suggest this to the maps team.) I also tried doing my own planning using paper maps, but basically Google did a better job. The trouble with doing my own long-distance route planning is that I can’t hold all the alternative routes in my head, so I end up committing to an initial section of route and then trying to find good routes forward from there instead of backtracking to consider alternatives. In particular, I don’t think I would have found the very nice route through the Chilterns via Flaunden on my own.

I looked at the journey planner at cyclestreets.net but that limits you to journeys of 30 km or less. OK for commuters, not so good for tourists. (But I quite understand why they have this limit—route planning is computationally intensive. They are doing fantastic work on very little in the way of funding.)

A route under the M4, according to the Sustrans mapping system, but not actually a cycle path.

I also looked at Sustrans, but their routes generally link centres of population, whereas the touring cyclist wants to avoid these places. They had almost nothing going my way, and what they had I was slightly suspicious of. I could have taken the Colne Valley Trail from Uxbridge to Rickmansworth (avoiding Harefield), part of route 61, but on the Ordnance Survey 1:50,000 sheet 176 this route is marked as a footpath, not a bridleway or cycle path. So I decided not to risk it. Sustrans also show a cycle route passing under the M4 northwest of Harmondsworth (see right), which I did take a chance on. This turned out to be a bad idea. The route petered out into a maze of recreational bridleways and footpaths filling the area bounded by the M4, M25, and a Harmondsworth industrial estate. After much searching, I found the path I wanted, but it’s a footpath, not a bridleway or cycle path, and moreover, it’s closed. So I had to backtrack and go through Harmondsworth after all. A mistake that cost me at least 10 kilometres.

Where Google Maps suggested a route that involved long sections of trunk road, I looked for nearby quieter roads, and added additional destinations accordingly. You can see these additional destinations in the pins on the map above:

Cambridge.
Steeple Morden. This avoids a long section of the A10 and A505. (Although I have cycled the A10 to Royston, and sections of the A505, these are not enjoyable roads.)
Bedmond. The small lanes though Bedmond in several directions are the nicest way to avoid the confluence of the M1, M10 and M25.
Golden Pot. This steers the route to the west, staying outside the M25 and passing through a lovely section of the Chiltern Hills around Flaunden. (But actually going through Golden Pot was a bit of a mistake; see below.)
Medstead. Avoids a busy section of the A32.
Warsash.
Basing Dean, and ...
Empshott. Without these two, Google Maps directs you along the A32 and A31. With them, the route goes east into a pleasant section of rolling hills around Froxfield.
Elstead. Otherwise Google prefers the A325 and A323. Even via Elstead, there’s still a section of the A324, but I couldn’t see any way to avoid this without going several kilometres out of the way, for example, via Ash Vale.
Staines.
Harefield. I wanted to find a different route back to Cambridge, for example through Hertford and Braughing, but getting from Staines to Hertford involved seemingly endless kilometres of northwest London, or else a big diversion. So I settled for going back on the same route I came out.

M40/A355/A40 junction: not so bad.

Big A roads are not always bad. I had been dreading this bit of A40 and A335 (see right) which seemed to be the only sensible place I could get across the M40, but in fact it was fine, with enormous lanes giving plenty of room for even the biggest of juggernauts to safely pass a cyclist.

By contrast, the worst road on the whole trip was this section of the B3349 from Odiham to Golden Pot. It looked innocuous enough when I was planning, but it was horrible: very busy, and too narrow for two lanes of traffic plus a cyclist, so queues of motor vehicles would build up behind me, and there was no shoulder or verge I could pull onto to let them past. Had I known I would have gone on one of the slightly longer alternatives, through Herriard to the west, or Long Sutton to the east.

Having got a route planned, I checked it against my Ordnance Survey maps to make sure it wasn’t directing me down footpaths or the wrong way down one-way streets. The result was a series of instructions, of which this is a typical extract:

170.2  L@ Village St              0.5
170.7  1x Holt End Ln             2.0
172.7  C- Trinity Rd              0.6
173.3  C- Trinity Hill            0.8
       MEDSTEAD
174.1  C- Church Ln               0.1
174.2  R@ Wield Rd                0.3
174.5  L@ Common Hill             1.0
175.5  Slight R@ Bighton Rd       1.2
       BIGHTON
176.7  C- Chalky Hill             2.5
179.2  Slight R@ Bighton Dean Ln  0.5
179.7  L@ Bighton Ln              2.9

I carried these instructions in my pocket and consulted them as I was riding along, so that I only had to stop to look at the map when I was confused or unsure or when the instructions didn’t seem to match the actual junction layout. The map I carried on the bike was the OS 1:250,000 South East England which conveniently covered the whole of the trip and generally had just enough detail to correct any navigational errors. But if I were doing it again, I’d use the 1:100,000 series maps, although I would have needed to take four of them.

This all seems like a lot of work, but that’s because with 220 km to cycle in a day, time spent on navigation becomes really significant. There were about 200 junctions on the first day’s route, and even if I had only spent half a minute reading the map at each junction, that would have been getting on for two hours! So preparing directions that I could read and follow while riding the bike was important.

In the CTC forum, commenter thirdcrank points out that long-distance cycle touring, and the navigational problems thereof, have become more difficult over the years: “the problem nowadays is often that the main roads are simply unpleasant for cycling. When Britain's road network was still unmodernised—up to the late 1960s, say, the A roads took the easiest route through the hills and as most freight went by train and there were many fewer cars, the journey from Manchester to Cambridge [250 km] would have been a doddle. You would simply have followed the signs for Nottingham, Peterborough, then Cambridge.”

A hand-held GPS device might be a good alternative to printed directions. I don’t have any experience with such a thing, but if it’s reliable and you can keep it from getting rained on, it’s probably a better solution because it can tell you when you make a mistake, and it won’t get confused by networks of nameless roads in the way that I did around Froxfield.

If you’re doing shorter distances, say 80–100 km a day, you can easily afford to figure out your route as you go along. As commenter pq says in the CTC forum, “Just remember that all you're doing is riding a bike around in your own country. It's not a big deal and there's no need to mount a military style operation to accomplish it.”

Arbury Carnival

gareth.rees@pobox.com — Sat, 13 Jun 2009 18:41:21 GMT

Today was the Arbury Carnival. Here's part of the procession:

Voting dilemma

gareth.rees@pobox.com — Wed, 03 Jun 2009 16:23:18 GMT

Like some of my friends, I’ve having trouble deciding who to vote for in the European elections. I agree in general terms with the European policies of both the Liberal Democrats and the Green Party. I’m leaning Green at the moment: I like the Green Party manifesto more than the Liberal Democrats manifesto (with exceptions, some of which I note below), mainly because of its greater detail and specificity.

The tactical voting considerations are also quite strong: it’s not likely that the Lib Dems can get a second seat in the East of England constituency (they’d need at least another 10% or so over their 2005 result) whereas the Greens might be able to get one seat (they might only need another 5% or so). However, there are some arguments the other way.

For the Liberal Democrats:

My Westminster MP, David Howarth, seems pretty good. He’s sound on several issues I care about (civil liberties, identity cards, climate change), he wrote decent replies to my letters, and he seems to be innocent of corruption. So I think the party deserves some kind of reward for his hard work.

Against the Greens:

I strongly object to their proposal for “An immediate halt to xenotransplantation, genetic manipulation and cloning of animals” [manifesto, page 28].
I don’t think their energy policy adds up: they plan “to campaign for an end to nuclear power throughout Europe, and against any new nuclear plants”, and they say that “current coal station schemes must be cancelled” [manifesto, page 11]. But that’s going to leave a big gap between energy generation and consumption, and I don’t believe that their proposals for renewable energy and increased efficiency are sufficient to fill the gap in time (based on the analysis in Sustainable Energy by David MacKay).
I don’t like their association with animal rights extremists. Their candidate Rupert Read is a “frequent participant in demos over the years at places such as Huntingdon Life Sciences”. I don’t approve of guilt by association, but some of the anti-HLS protesters are really vile people and that’s hard to set aside emotionally.
I don’t like their plan for the NHS to pay for complementary and alternative medicine [§HE300].

However, that’s a fairly small list of objections, and it’s not likely that the Greens will actually have the power to implement any of the things I object to. So maybe I should vote for Mr Read despite my worries.

BNP press office on mixed-race marriages

gareth.rees@pobox.com — Tue, 02 Jun 2009 20:18:49 GMT

The Peter Burkinshaw thing reminded me of this e-mail exchange I had a few years ago with “Dr. Phill Edwards” of the BNP.

From: gareth.rees@pobox.com To: pressoffice@bnp.org.uk Subject: RE: Mixed-race marriages Date: Fri, 11 Jun 2004 22:43:53 +0100

I see from your web site <http://www.bnp.org.uk/faq.html#mixedrace> that you are against mixed-race relationships. Is your view a matter of conscience or do you propose to legislate to ban or restrict mixed-race relationships? I couldn't find the answer on your web site.

From: pressoffice@bnp.org.uk To: gareth.rees@pobox.com Subject: RE: Mixed-race marriages Date: Sat, 12 Jun 2004 04:43:24 -0400

Our web site makes it clear why we don't approve of race mixing. We would not ban or legislate against them but we would ban the promotion of mixed race liasons and allow people to make their own choice without being persuaded that it's "cool" to have black/white partnerships. Adverts, soaps, TV dramas always show a positive picture of mixed race liasons (as they do with queers) and, of course, anyone who disaproves in public is immediately condemned as a "racist" (or "homophobe"). Apart from the obvious problem in multiracial Britain of chance encounters by people from different races, I believe that if people are allowed to make up their own minds, uncluttered by media conditioning, they will choose "partners" from their own racial group. It's a matter of evolutionary biology.

Dr Phill Edwards BNP National Press Officer

From: gareth.rees@pobox.com To: pressoffice@bnp.org.uk Subject: Re: Mixed-race marriages Date: Sat, 12 Jun 2004 15:56:17 +0100

Thank you for your reply. I have a couple of questions.

You propose to legislate against the promotion of mixed-race relationships in television programmes and in television advertising?

Do I understand you correctly? What about newspapers, magazines and other printed materials? What about plays?

For example, under your proposed legislation, would it be permissible to stage a play of Othello? To show a movie of Othello on television? To publish the text of the play in print?

From: pressoffice@bnp.org.uk To: gareth.rees@pobox.com Subject: Re: Mixed-race marriages Date: Sat, 12 Jun 2004 15:29:23 -0400

I'm sure you are aware that at the present time there are a number of quangos which regulate adverts (ASA), print (PCC), content of TV soaps, plays etc - so there is legislation currently in place to regulate these outlets. We would increase this to prevent the promotion of inter racial liaisons and the activities of queers using the existing laws. After all, if the regulators currently allow offensive material to be printed, broadcast etc then we would improve the situation. We would also remove restrictions on what can and cannot be said in order to allow free and open debate about topics which are now censored eg the desirability or not of multiculturalism and multiracialism. There would be exemptions when appropriate eg Othello. Me - I would like to see the Black and White Minstrel show back on the BBC. Fox (the BBC boss who pulled it) said it was the most popular TV show at the time yet it had to go. So much for the rights of the native white people of the UK

What a Burk

gareth.rees@pobox.com — Tue, 02 Jun 2009 14:37:11 GMT

The Cambridge Cycling Campaign asked all the candidates standing in the local elections for their views on issues that are important to people who cycle in Cambridge.

Here are the answers given by Peter Burkinshaw, standing for the UK Independence Party in East Chesterton ward:

CCC: Question 1. There is a major shortage of cycle parking all around the city. Cycle theft is over 10% of all reported crime in the County. Do you have any suggestions for locations for cycle parking? Would you be willing to see a very small proportion of on-street car parking being replaced by on-street cycle parking in your ward? How will you progress towards a situation where every resident and every worker in each ward can keep a bike safe?

Burkinshaw: No

CCC: Question 2. Do you support our view that traffic policing (including fining of cyclists without lights or using pedestrian-only pavements) should become a greater police priority?

Burkinshaw: No

CCC: Question 3. We believe that 20mph should be the norm for local streets in residential areas (as distinct from main connecting roads). 20mph would: greatly encourage walking and cycling; improve the quality of life in an area for residents; and would not delay car journeys significantly (because only the start/end of a journey would be affected). Do you agree that 20mph should become the norm for local streets in Cambridge and surrounding villages?

Burkinshaw: No

CCC: Question 4. If the County Council's proposed Congestion Charge goes ahead, it is likely that the associated up-front money that would be received from the government to support prior improvements to public transport and cycling would be of the order of some £500m spread over five years. This is roughly ten times the amount the County currently receives for transport. If the scheme goes ahead, what would be your priorities for use of this up-front money?

Burkinshaw: To reduce the Council tax

CCC: Question 5. The Haling Way / Penny Ferry path is part of a national cycle route. Its entranceway is currently being remodelled, at considerable cost. We believe that a diagonal entrance into the path should have been created, rather than the current ‘wiggle’ round a blind corner and use of the pavement. This has not been done because it would involve removal of a few more car parking spaces. Do you feel that this cycle route should have a proper entrance for cyclists, even if it means the loss of three or four parking spaces?

Burkinshaw: No

CCC: Question 6. Do you have any other general cycling-related comments or points? And what support have you given for cycling and walking, or sustainable transport more generally, in the past?

Burkinshaw: Provision for cyclists is already adequate. Please remember that motorists are the people who pay to use the roads whereas cyclists are "freeloaders". They are entitled to use the roads but not disproportionately. If everyone cycled, as you suggest, there would be no roads to ride on.

Update 2009-06-04: He got 220 votes and came fifth in East Chesterton.

Punctures

gareth.rees@pobox.com — Sun, 31 May 2009 20:27:38 GMT

It seems to be the season for punctures. On Saturday I was cycling into town when I spotted a cyclist standing disconsolately by the road, staring at his back wheel. So I stopped and fixed his puncture for him.

Then on Sunday I went out with the touring group of the Cambridge Cycling Club. We were supposed to be going to Grafham Water, but we had two punctures, so we ran out of time and had to come back. And then we had another two punctures on the way back.

Puncture 1. Between Bourn and Caxton

Puncture 2. In Great Gransden

Puncture 3. In Papworth Everard

Puncture 4. In Longstanton

Couldn't have happened on a nicer day, though.

Pillars of the community

gareth.rees@pobox.com — Tue, 26 May 2009 23:23:30 GMT

The 2009 Étape Caledonia: cyclists waiting for the road to be cleared. Photo by Colin Frame. (Licence: CC-by-nc-sa.)

Here is a cri de coeur from Matt Polaine:

Thanks to the feeble government attempts to encourage an increase in cycling, road layouts, both old and new, continue to be peppered with lethal designs for cyclists, the police continue to be indifferent to reports of road rage or just plain terrible driving towards cyclists and one often has to really push for a prosecution. Even if a prosecution does find its way to the courts, the judges take ‘Sorry Mate I Didn’t See You’ as a water-tight excuse for ploughing into a cycling group, on a straight road, in daylight. [...] This hate of cyclists extends to whole UK towns now.

I don’t necessarily endorse everything in Polaine’s rant, but that’s not really the point—it’s the feeling of being a member of a minority under siege that he’s expressing, and it’s one that I share from time to time. On a sunny weekend, cycling is so enjoyable that it would be easy to forget about all the unpleasantness if you weren’t reminded of it every day in the news.

The story of the sabotage of the 2009 Étape Caledonia is really quite extraordinary. The Étape Caledonia was the first British cyclosportive to be held on roads closed to traffic. The Independent reported:

An audacious act of sabotage threw one of Britain's biggest cycle races into chaos yesterday when a protester, presumably angered by road closures for the annual 3,500-cyclist Etape Caledonia, scattered the road with grey carpet tacks, puncturing hundreds of tyres. [...] The sabotage throws into doubt the ability of the UK cycling organisations to host the type of closed-road events common on the continent.

The same day that the Étape Caledonia was sabotaged, several roads in the centre of Manchester were closed for the Great Manchester Run. As far as I know no-one attempted to injure Haile Gebrselassie or any of the other runners.

So what kind of antisocial thug would take out his frustrations at the road closure by cold-bloodedly attempting to injure the 3,500 particpants in the cyclosportive? Here’s the Daily Record:

Alex Grosset, 62, was arrested at his home in the early hours of yesterday morning. He is expected to appear before Perth Sheriff Court today to face reckless conduct charges relating to thedisruption of the Etape Caledonia event through Perthshire on Sunday. Grosset is the chairman of the Rannoch and Tummel Community Council and a member of the local Rotary club.

So this pillar of the community considers the lives of cyclists to be so worthless that it was reasonable to risk them to make a political protest.

In other news, Boris Johnson (Mayor of London), Andrew Adonis (a Minister of State in the Department for Transport), and some others, were nearly killed by a reckless lorry driver. Now I disagree with Mr Johnson on nearly every political issue, but I still wouldn’t like to see him hurt by a heavy goods vehicle. The incident was captured on a private security camera monitoring the Dunbar Wharf development:

Nic Price (aka “Beatnic”) was an eye-witness:

The mayor and an entourage—about 10 cyclists or so—were looking at options for a new cycle route - as a curious fellow cyclist I tagged along as I was on my way in to work in Canary Wharf following the same route. The back doors of the lorry flew open as it overtook us and the bolt on the right-hand door picked up a parked car through its front windscreen and swung it round at head-height, brushing past a few of the cyclists and then landing it back on all four wheels a little further down the road.

I think every cyclist will recognize something here: there’s a kind of psychological compulsion to overtake cyclists that affects a (luckily very small) minority of drivers. This compulsion leads drivers to make extraordinary efforts to get ahead of cyclists even when it can’t possibly do them any good, such as racing to overtake when coming up to a queue of stationary vehicles, or to a traffic light that’s red. Combined with narrow streets this compulsion can be deadly, as these drivers make desperate attempts to overtake cyclists when approaching a gap that’s too narrow for both of them.

What can we do? We could identify these drivers and ban them—but I think that as a society we’re clearly not willing to do this. Something we could try instead would be to redesign roads so that drivers of motor vehicles no longer feel compelled to make these dangerous manoeuvres. That means taking out width restrictions, central reservations, and the kind of on-road parking that you can see in the video, and using the space so gained to put in wide cycle lanes or separated cycle paths. It’s the kind of project that’s esily within the powers of an advanced industrial society like ours. If the Dutch can do it, then so can we.

But we’d have to have politicians who valued the lives of cyclists more than their own driving convenience, unlike the councillor for Rannoch and Tummel. And I don’t see how to get there from here.

Bonk

gareth.rees@pobox.com — Tue, 26 May 2009 12:39:54 GMT

Since the weather was so nice on Sunday, I went on this ride with the CCC in the morning (that's me in black in the photo), and then on this ride with the CTC in the afternoon. I failed to eat enough lunch, and even more foolishly failed to carry any food with me, so I completely ran out of glycogen on the climb up to Barley in Hertfordshire. Fortunately the Chequers pub was serving food and after a bowl of crumble and custard I had enough energy to make it to tea in Litlington. Just over 100 miles for the day (including three ascents of Chapel Hill near Barrington!), but I need to improve my eating skills.

Birds

gareth.rees@pobox.com — Mon, 25 May 2009 17:51:41 GMT

The starlings in the neighbourhood are fledging at the moment. The fledgelings are still rather poor fliers—they make short flights from the perch where their parents are waiting and then turn around and go back. When the parents look for things to eat, the fledgelings follow them around and pester them for food. There are only about four or five families, but they are very noisy.

The blackbirds in the pyracantha by the door have not yet flown the nest, but it can’t be long now. There are two chicks, but they are rather camera-shy. In the photo I think you can just about see the beak and eye of the one on the left, but the one on the right is head-down.

Not making the best argument

gareth.rees@pobox.com — Thu, 21 May 2009 23:00:48 GMT

Charlie Brooker writes about the BNP’s party political broadcast:

But there's more to the advert's failure than its hideous use of colour schemes. Every aspect of it is bad. The framing is bad. The sound is bad. The script is bad. For all their talk about representing the Great British Worker, when it comes to promotional material, the BNP can't even represent the most basic British craftsmanship.

Is this a dangerous approach to be taking? If one of your arguments against the BNP is that they are rubbish at making television commercials, then on that day that the BNP have enough money to make something a bit more professional, you end up looking a bit refuted.

On the other hand, if you appear to be taking them seriously, you end up loaning them a bit of your credibility, at least as some kind of worthy opponent. So maybe the best rhetorical strategy is ridicule?

Private Eye often chooses to criticize someone by contrasting their pious words with their shameful deeds, or their words at one time with their words at another. For example, from this week:

“Sir Fred Goodwin should not be counting on being £650,000 a year better off. It’s a huge amount of money for nothing. The Prime Minister has said it is not acceptable and the government will take action. [...] Sir Fred should not be counting on this. It might be enforceable in a court of law this contract but it's not enforceable in the court of public opinion and that's where the Government steps in.” — Harriet Harman, interviewed on BBC, 1 March

“People have claimed in good faith under the system. MPs make the claim under the system and it’s for the House of Commons fees office to decide whether it comes within the rules. MPs make their claims under the rules and the money is paid out only if they are satisfied that the claim is within the rules.” — Harriet Harman, interviewed on BBC, 8 May

It makes for effective satirical comment to have the accused condemn themselves with their own words. But when the tactic is used week after week it starts to give the impression that the fault being criticized is purely the hypocrisy of saying one thing and doing another, and if only the person had got their story straight they would have been fine.

A question

gareth.rees@pobox.com — Mon, 11 May 2009 19:41:01 GMT

In light of the following figures from the National Audit Office's report Improving road safety for pedestrians and cyclists in Great Britain —

Mode	UK fatalities per 100 million passenger km average 1997–2006
Pedal cycle	3.404
Pedestrian	4.421

(suggesting that on average cycling is about 20% safer than walking a similar distance in the UK) — why do you think they picked a cover photo in which the cyclist is wearing a helmet but none of the pedestrians appear to be wearing any protective equipment?

Ride to St Neots, Keysoe and Caxton

gareth.rees@pobox.com — Mon, 11 May 2009 10:15:49 GMT

Sunday's bike ride, at the Cambridge CTC blog.

Improving road safety for pedestrians and cyclists in Great Britain

gareth.rees@pobox.com — Fri, 08 May 2009 22:25:09 GMT

The National Audit Office has just published a report for the Department for Transport, Improving road safety for pedestrians and cyclists in Great Britain.

Here are the headline figures:

Mode	UK fatalities per 100 million passenger km average 1997–2006
Air	0.000
Water	0.024
Bus or coach	0.029
Rail	0.032
Van	0.086
Car	0.269
Pedal cycle	3.404
Pedestrian	4.421
Motorcycle	11.144

The most obvious problem with this report is the persistent confusion between absolute casualties, and casualties normalized by population and by distance travelled. In some cases I simply can’t tell which is meant, for example on page 10:

“The Department has targets to reduce by 2010 the numbers of people killed or seriously injured by 40 per cent, of children aged 0 to 15 by 50 per cent and slight injury rates per 100 million vehicle kilometres by 10 per cent compared with the average between 1994 and 1998.

Which of these three targets are absolute and which normalized to population and distance? I haven’t a clue. This matters because effective strategies are going to differ according to which measure is being targeted. If the target is simply to reduce the absolute number of casualties from cycling, the simplest approach would be to ban cycling altogether: the target would be achieved automatically. If such a ban proved politically impossible, then making cycling substantially more inconvenient (for example by imposing a registration scheme or other tax) would probably have a similar effect. But can this really be the kind of “safety” that the Department for Transport has in mind?

This confusion between absolute and proportional numbers comes to the fore in the section on international comparisons, where this claim is made:

“The United Kingdom was fourth highest out of 14 European nations in 2006 for the least number of cyclist deaths per head of population.” [page 12]

And backed up by this table on page 36:

Rank	Country	2006 cyclist fatality rate (per million population)
1	Malta	0.0
2	Spain	1.7
3	Greece	1.9
4	UK	2.5
5	France	2.9
6	Sweden	2.9
7	Portugal	3.8
8	Finland	5.5
9	Denmark	5.7
10	Austria	5.8
11	Belgium	8.8
12	Estonia	9.7
13	Czech Republic	10.7
14	Hungary	15.2

This seems to me to be a really misleading table to include in the report. Because these numbers are not normalized to distance travelled, the table is primarily telling you about the amount of cycling in each country, not the relative safety of cycling in that country. People in the UK hardly cycle at all—the report estimates an average of 39 miles per year per person—so it is not surprising that the number of fatalities is low too.

And where are the Netherlands? Using the figures from 2007 (see below) of 147 fatalities and a population of 16.3 million, I get a figure of 9.0 fatalities per million population. So why aren’t the Netherlands in there between Belgium and Estonia? (This could be because the Netherlands were dilatory in reporting road casualty figures for 2006 to the European Road Safety Observatory—in ESRO’s Annual Statistical Report 2008 the latest figures for the Netherlands are from 2003. But it would have still been fairer to include the most recent datum from that country, with a note about what year it’s from.)

The appropriate table would have shown the data normalized by distance travelled. Here’s my best attempt to generate such a table:

Country	Year of fatality statistic	Number of cyclist fatalities	km cycled per person per day (2000)	Population in millions (2007)	Cyclist fatalities per 100 million km cycled
Malta	2007	0	–	0.4	0.0
Netherlands	2007	147	2.5	16.4	1.0
Ireland	2003	10	0.5	4.3	1.3
Sweden	2007	33	0.7	9.1	1.4
Finland	2007	22	0.7	5.3	1.6
Denmark	2007	54	1.6	5.4	1.7
Germany	2006	486	0.9	82.3	1.8
Belgium	2006	92	0.9	10.5	2.7
France	2007	142	0.2	63.4	3.1
Austria	2007	37	0.4	8.3	3.1
UK	2007	136	0.2	60.8	3.1
Italy	2004	296	0.4	59.1	3.4
Greece	2007	16	0.1	11.2	3.9
Spain	2006	75	0.1	44.5	4.6
Portugal	2007	34	0.1	10.6	8.8
Czech Republic	2007	116	–	10.3	–
Estonia	2007	13	–	1.3	–
Cyprus	2007	3	–	0.8	–
Latvia	2007	18	–	2.3	–
Hungary	2007	158	–	10.1	–
Poland	2005	603	–	38.1	–

Sources:

Number of cyclist fatalities — European Commission. EU energy and transport in figures, 2009.
km cycled per person per day — Figure 2 of John Pucher and Ralph Buehler (2008). “Making cycling irresistible: lessons from the Netherlands, Denmark and Germany”. Transport Reviews 28:4:495—528. DOI: 10.1080/01441640701806612.

I’m not particularly happy with using these statistics because they are so out of date, but they are the best I can find for the moment. If you know of a better or more up to date source, please let me know. Pucher and Buehler source them from EU energy and transport in figures, 2000, and that’s not online. More recent issues of the same publication which are online don’t report this statistic. (However, the estimate for the UK — 0.2 km per person per day — is a good match for the National Audit Office’s figure of 0.17, so maybe they are not too bad.)

Update: I e-mailed John Pucher to ask if he knew of more recent statistics for distances walked or cycled in the EU countries, and he wrote back, “The [European Commission] no longer publishes those estimates of distances walked and cycled, unfortunately. So the only alternative is to to get the info from individual countries, which we did for [the Netherlands], [Denmark], Germany, UK and USA, but it's too much work to do it for all countries.”
2007 population — Eurostat. European Union: Total Population.

You can see that the normalized figures are not nearly so flattering to the UK. Is this just plain incompetence, or a deliberate attempt to mislead?

So much for the statistical side of the report. What about the recommendations?

Well, I was going to tell you about them, but I fell asleep. There’s absolutely nothing concrete to get a grip on, nothing that will result in any action, nothing that will make any actual difference to someone cycling or walking. Here are some highlights (page 7):

The Department [for Transport] should set targets that report separately the numbers of people killed and those seriously injured
The Department should complete by Autumn 2009 [...] its work on assessing the usefulness of Hospital Episodes Statistics
The Department should assess whether and how it can use other data [...] to improve the reporting of trends in road safety.
The Department should allow a lead time before projects commence so that local highway authorities can undertake sufficient consultations
The Department should require local highway authorities to adhere to prescribed evaluation standards.
The Department needs to develop an explicit strategy which [...] develops key indicators to assess how well it works with other bodies

Most of the recommendations are completely non-actionable — “considering”, “engaging”, “educating” and “influencing” are the key verbs — and the report admits that the Department’s education campaigns have unmeasurable results (“there is no direct evidence of the contribution that the Think! campaign has made to reducing casualties”).

So thank you, National Audit Office. Next time I get run down by a bus, I will remember how you recommended that the Department for Transport develop an explicit strategy for developing key indicators for assessing how well it works with other bodies.

Manifold: Time

gareth.rees@pobox.com — Wed, 08 Apr 2009 23:07:10 GMT

Manifold: Time by Stephen Baxter (Voyager, 2000).

Reid Malenfant is a maverick billionaire who is determined to launch a manned space mission to mine the asteroid belt. The bureaucrats of NASA and crazy anti-space environmentalists try to thwart him with their weapons of red tape and government regulation, but Malenfant is too mavericky for them, and launches his hastily thrown-together rocket in the nick of time—with Malenfant aboard, of course—leaving the bureaucrats standing helpless by the launch pad.

The standard set of space advocacy soundbites pile up as Malenfant pitches his plans to investors—“If we succeed [in colonizing space], we will live forever. The alternative is extinction.”—“[A solar] system’s resources ... are useless for any other purpose, and are therefore economically free to us.”—“A metallic-type near-Earth object would be worth, conservatively, trillions in today’s market.”—“If you reach a C-type, a carbonaceous chondrite, full of water and organic compounds, you can ... throw bags of water and food and plastics back to Earth orbit, where they would be worth billions in saved launch costs.”—“Because of NASA’s safety controls and qual[ity] standards it takes years and millions of dollars to prepare your payload for flight.”

It’s hard for me to tell to what extent Baxter means this to be taken seriously. The maverick space entrepreneur who could easily conquer space if it weren’t for those pesky pen-pushers at NASA is such a cliché that surely no-one can now play it straight? The fallacies of space advocacy are sufficiently shopworn that an intelligent writer of science fiction with a background in science or engineering should have no trouble seeing through them. Later in the book all Malenfant’s plans come to ruin, so maybe the idea is to undermine these clichés by showing their failure? On the other hand, the early sections of the book are told with such a poker face that there’s no indication that the author has any kind of ironic or satirical attitude to this material. Indeed, he says in an interview that he never writes in ironic mode:

Basically I try not to employ irony, allusive or recursive or otherwise, just as I try to keep out of the forefront of my mind all the levels of metaphor that come with any piece of fiction. I try to get fully immersed in the fiction; I'm not interested in writing elaborate jokes. I'm just trying to tell as compelling and honest a story as I can, with the tools I have at my disposal.

The annoyance factor, already at dangerous levels, goes off the scale when one of the characters introduces the Doomsday argument. I’ll summarize the argument here:

Suppose that you have an a priori estimate for the distribution of the number of humans who will ever live.
Suppose further that you are a random sample uniformly selected from all the humans who will ever live.
Note that you are about the 60 billionth human to live.
You can use (2) and (3), plus Bayes’ theorem, to refine your estimate of the distribution in (1). Plugging in plausible distributions for (1), we find that (as Wikipedia puts it) “it is unlikely that more than 1.2 trillion humans will ever live.”

The argument as stated is logically correct: if all the assumptions were true then the conclusion would follow. But assumption (2) is wrong, so the conclusion does not in fact follow. So the main rhetorical trick in presenting the argument is to obfuscate the role of assumption (2), either by skipping it or by making an analogy with a statistical situation in which the sample is genuinely random. Baxter neatly pulls off the latter bait-and-switch by making an analogy with picking balls from a box. Note the phrase I’ve emphasized in the first paragraph.

Cornelius reached under the table and produced a wooden box, sealed up. It had a single grooved outlet, with a wooden lever alongside. ‘In this box there are a number of balls. One of them has your name on it, Malenfant; the rest are blank. If you press the lever you will retrieve the balls one at a time, and you may inspect them. The retrieval will be truly random.

‘I won't give you the opportunity to inspect the box, save to draw out the balls with the lever. But I promise you there are either ten balls in here—or a thousand. Now. Would you hazard which is the true number, ten or a thousand?’

‘Nope. Not without evidence.’

‘Very wise. Please, pull the lever.’

Malenfant drummed his fingers on the tabletop. Then he pressed the lever. A small black marble popped into the slot. Malenfant inspected it; it was blank. Emma could see there was easily room for a thousand such balls in the box, if need be.

Malenfant scowled and pressed the lever again.

His name was on the third ball he produced.

‘There are ten balls in the box,’ said Malenfant immediately. [page 45]

Why is this so annoying? It’s interesting to think this through, because I don’t normally object to science fiction that uses wrong ideas or incorrect science (nearly all of the genre). But suspension of disbelief can’t be compelled, so there needs to be some concession to the fact that the science is wrong. For example, Greg Egan’s novel Quarantine uses the idea that human consciousness causes the collapse of quantum superposition. This was a genuine piece of speculative quantum physics—the idea was discussed by physicist Eugene Wigner in his 1961 essay “Remarks on the Mind-Body Question”—but I think it’s safe to say that no serious physicist believes it now, and certainly Egan doesn’t. So the novel has some work to do in getting the reader to the point where suspension of disbelief is possible, and Egan accomplishes this by describing a series of observations and experiments whose results can only plausibly be explained in terms of the effect of human consciousness on quantum superposition. So when the novel asks us to consider the “consciousness causes collapse” theory, there’s a narrative payoff: never mind that it’s nonsense in the real world, in the word of Quarantine it provides an explanation for a series of mysterious events. So the reader is primed to suspend disbelief and rewarded for that suspension.

At least, it worked for me. There are clearly personal differences in the kind of nonsense that readers are willing to entertain. For example, nickbarnes stopped believing in Egan’s novel Schild’s Ladder at the introduction of the idea that the phase transition to the fictional “novo-vacuum” of the novel should propagate at half the speed of light. This didn’t bother me; having swallowed the novel’s idea of the laws of physics being explained by transformations of an underlying graph, the idea that some effect could propagate along the graph at a rational fraction of the speed of light wasn’t much of a gnat to strain at. But for another reader it was. So Egan could have improved his novel for at least one reader by taking more care to justify this piece of fictional physics.

But back to Time. The use of the Doomsday argument annoyed me because Baxter makes no attempt to justify the argument within the fictional world of the novel. The argument could be made plausible within a science-fictional context by showing us a series of anthropic experiments that ratify the randomness assumption, or by hand-waving some science-fictional rationale for it. But there’s nothing like that; the argument is just dropped in there with no clue as to its status within the fiction. There’s no narrative payoff from accepting it. So are we meant to interpret it as a truth of the invented world? Or as evidence of the craziness or untrustworthiness of the character who makes the argument and the credulity of the (many) characters in the novel who are convinced by it? Or could Baxter even believe it himself? Reviewer Dave Langford was so troubled by the last of these theories that he contacted Baxter to check! “I was relieved to learn (by personal enquiry) that Baxter himself doesn't take the argument seriously”.

However, the space entrepreneurship and the Doomsday argument plots soon drop out of the novel as the space travellers find a magic portal which gives them a package tour of a far future in which human civilization survives alone in a decaying universe. This is potentially a powerful vision, of intelligent life struggling to gather enough energy to power its existence while everything runs down to the ultimate heat death:

‘... the smallest, longest-lived dwarf [star]s can last for maybe a hundred billion years, a lot longer than the sun. But the interstellar medium is a finite resource. Sooner or later there will be no more new stars. And eventually, one by one, all the stars will die. All that will remain will be stellar remnants, neutron stars and black holes and white dwarfs, slowly cooling.’ He smiled, analytic. ‘Think of it. All that rich, complex dust and gas we saw before, locked up in the cooling corpses of dead stars ... And then, this.’ Cornelius pointed. ‘The wreck of the Galaxy. Some of the dying stars have evaporated out of the Galaxy. The rest are collapsing into the great black holes—those blisters you see in the disc ... Those stars are small and cold. Designed for longevity. Their worlds must be huddled close—probably gravitationally locked, keeping one face in the light, one in the dark...’ [pages 197–199]

It could be a powerful vision—but it’s presented by Cornelius, the same character who tried to fool us with the Doomsday argument. So it loses much of its force by being narrated by such an untrustworthy (and annoying) character.

In parallel with the deep futorology, Time develops a plot about superintelligent children who begin to be born to ordinary parents. These children inspire fear and hatred in the general population, and they are taken away from their families and concentrated in orphanages. In what looks like a reference to the stolen generations of native Australians, the viewpoint children are taken to an institution in Australia where they are brutalized and starved. This strand of the story is an indictment of our xenophobia and fear of strangers, right? Except that it turns out that these children are planning the destruction of the universe. So is the idea that it was right to fear and distrust them? What’s going on here?

There’s a clear pattern in all these problems: Baxter completely lacks control over his material in this novel. He has potentially powerful themes and ideas, but handles them so inconsistently that they lose all their power. The novel starts as if it’s a story of ambitious space entrepreneurship—but it comes to nothing because the universe ends. Is this Baxter satirically undermining a science fictional cliché? Or is it just that he likes to write about space entrepreneurs? He puts the powerful central vision of the future of humanity in the mouth of a character whose credibility he has destroyed via the Doomsday nonsense. Is this meant to undermine the vision, to show how little faith we can put in this kind of futurology? The brutal treatment of the cuckoo children appears at first to be an impassioned criticism of xenophobia, but then the effect is completely undermined as the children turn out in fact to be a threat to the survival of the human race. Baxter’s depressing view of humanity as wasteful, fearful and violent is one for which you can find plenty of ammunition in the real world. But result is an incoherent book: you can’t have half your plot driven by “oh no! humanity must be saved from extinction!” and the other half driven by “humanity is too vile and stupid to be worth saving”.

Baxter needs to think a bit harder about what kind of aesthetic effect he’s trying to achieve, and then to deploy his considerable resources of invention and speculation so that they all face in the same direction.

Roberts on Incandescence

gareth.rees@pobox.com — Fri, 20 Mar 2009 15:30:43 GMT

This is a commentary on Adam Roberts’ review of Greg Egan’s novel Incandescence in Strange Horizons (2008-06-06). There are some spoilers for the novel.

Roberts teaches English literature at the University of London, writes and reviews science fiction, and blogs at the Valve and elsewhere. He’s very insightful about the use of language, for example in this hilarious satirical review of Neal Stephenson’s novel Anathem. However, in his review of Incandescence he appears to declare a lack of interest in science itself:

[I] was less intrigued by Zak and Roi’s interminable toing and froing with stones and springs inside the Splinter, to which adheres the odour of fourth-form school physics labs.

(I’m interpreting “odour” as a signifier of disgust here; cf. George Orwell’s essay “Such, such were the joys”.)

This lack of interest makes his review very unsatisfactory. In particular, I think it leads him to misidentify the main subject of the novel as being “finding stuff out”, that is, the process of scientific discovery:

[The characters] come across as ciphers through which Egan rehearses the process by which scientists undertake experiment after experiment in order to move closer to the truth.

It seems to me that the main subject of the book is the general theory of relativity itself, not just the process of its discovery, and the process by which Egan’s characters discover the theory is of interest because of its complete difference from Einstein’s discovery, not because it’s a “rehearsal” of similarity.

Egan’s novel is deliberately constructed so as to the dramatize the theory of general relativity, by contriving a situation in which accurate understanding of the theory makes the difference between life and death for a whole civilization. It’s also a demonstration of the universality of physical laws. Einstein’s discovery of general relativity was driven by mathematical hypothesis (what theory do we get if we assume the equivalence of accelerating and gravitational frames of reference?) and only verified by painstaking observation after it was formulated, once people knew what to look for. But in Incandescence, the characters, living in a much more curved region of spacetime than our own, are driven to discover general relativity by direct observation of the dynamics of orbits. Completely different contexts of discovery lead to the same underlying physics.

Everything else in the novel forms the supporting framework for this dramatization, and the difficulty of making it work at all requires Egan to write in the ways that Roberts so dislikes. Because the material is complicated, it’s necessary for the text to be “dry” (but it’s not “unengaging” if you’re interested in the subject). Because there’s so much theory to cover, it’s necessary for the other elements of the story to be streamlined and simplified, hence the flat characterization. Because the setup is so very contrived (civilization in orbit closely around a black hole; civilization discovers gravitation from first principles in a very short time), there needs to be a backstory explaining how it came to pass.

This is why Roberts’ remark that the “novel as a whole feels like a neat-oh short-story idea that has been stretched” is so wrong: Egan barely manages to compress his material into the novel as it is. (Compare with Gravitation by Misner, Thorne and Wheeler, which is 1,215 pages long!)

Roberts’ failure to appreciate what the novel is about leads him to flail about looking for details to criticize, and this results in a couple of embarrassing appeals to grammatical shibboleths:

It’s a shame that Egan seems innocent of the proper use of the subjunctive mood (“if this world was a bacterial graveyard ...” p. 102) or the inelegance of ending sentences with prepositions (“...the abundance that she was used to” p. 40)

Both of these constructions are perfectly fine in modern informal English. Indeed, even according to the traditional rules for “if” clauses, the subjunctive is supposed to be reserved for cases where the supposition is counterfactual (see The American Heritage Book of English Usage). Egan’s use is correct according to this rule, since the character is supposing the world in question to be a bacterial graveyard. Of course, in real English usage this rule is not observed, as the AHBEU goes on to point out. And see Mark Liberman on sentence-final prepositions.

However, most of Roberts’ other criticisms are in fact perfectly justified—Egan’s characters really are weak, the dialogue is very dry, there really is a lot of infodumping—but I think they show a lack of insight into what Egan is trying to do. These criticisms would be spot-on when analyzing a novel about character or human relationships but they completely miss the target here. Egan is not writing about character or human relationships, he’s writing about the laws of physics.

Of course Roberts cannot be made to enjoy fiction about the laws of physics: if he found the subject boring in fourth form he’s going to find it boring now. So why did he attempt a review of a book that he was so unsuited to appreciate? I guess that this is just the kind of mismatch that happens all the time in reviewing (sometimes the mismatch is deliberate, to stir up controversy, but I think most often it’s accidental, and that’s probably the case here).

In my own review of Incandescence (2008-08-25) I dismissed the whole question of character and plot in Egan’s work:

Egan has little interest in the idea that science fiction is about exploring human responses to changes in technology and society: his characters are often post-human or non-human, and rarely particularly engaging or sympathetic [...] After all, there are plenty of other authors writing about 20th-century people in space.

But now I wonder if there might be some way to bridge the gap? The visible construction lines in Incandescence suggest that such a book is presently beyond the powers of the genre, but maybe now that Egan has blazed the trail other writers may be able to follow?

Greg Egan wrote his own response to Roberts’ review, “Anatomy of a hatchet job” (2009-03-09) which makes some similar points to mine, but is perhaps a bit unwise in the strength of its rhetoric. I doubt that Roberts set out deliberately to write a hatchet job; his review is probably a genuine attempt to engage with the novel that’s gone astray.

I think Egan can have faith in his core readership to understand what he’s trying to achieve, so he does not need worry too much about reviewers who don’t quite get it. Instead, he could use this kind of review as a clue as to how to broaden his appeal (if that is indeed something he is interested in). Maybe Roberts is out of reach, at least for the moment, but I’m sure that there’s a wider readership that could be turned on to his kind of fictional exploration of science, given an appropriate sweetening of character, language, and plot.

But I would miss the stark, spare, rigour of Incandescence.