Today I created an image (which I will link to later) of Visio being on crack (i.e. not displaying text or icons on its menus), and mentioned the URL only three times on IRC. A few hours later, I was looking through my web server logs, and somehow Google had discovered the file! As far as I know, it's not linked to anywhere (including any type of automatic index created by Apache). My two hunches are 1) Someone's making logs of the channels I'm in available on his/her web site, or 2) Google's toolbar is reporting URLs back to Google to be spidered later.
Come to think of it, it's probably someone using the Google toolbar with the PageRank(TM) meter turned on. When Google got a request for the PageRank(TM) of an unrecognized URL, it logged that URL to be spidered later. This has interesting privacy implications, because a secret URL you share with someone might not remain one (think naked photos and Google's image search).