Robots.txt, mediawiki and Google Sitemap October 13, 2005
Posted by Andy Roberts in : learning, internet , add a commentI used to have my ukcider mediawiki excluded from most search engines through a robots.txt file which looked like this:
User-agent: * Disallow: /wiki/
but then I decided I’d like to have another go at allowing the Googlebot to index some of the really useful content which has been building up there recently, so I removed the robots.txt file for a few days and monitored carefully.
What appears to be happening is that the googlebot visits about once per day and spiders a little further down into the Wiki each day, but using up an ever increasing amount of bandwidth as it does so - not good. So the list of french cider producers can already be searched for, but the Asturian Campsites - not as yet.
My own webstats and research told me that Googlebot can get caught up in a wiki site, spidering all of the previous versions, page history, user contributions and so on, and if you are paying for the remote hosting then this needs to be avoided. So rather than disallow /wiki/ I’ve disallowed “oldid” and “contributions” for now, and maybe I’ll tweak it a bit later or go fishing for the definitive mediawiki (not pretty URLs) robots.txt configuration. Meanwhile in my travels, I came across a reference to Google sitemaps which should allow me to tame the over eager googlebot some more. I’ve included data to the effect that the site is updated weekly, which should help towards my goal of having deep-linked pages listed on search results without having all the bandwidth used up by spiders.
Googlebot is not the only search engine spider, there are many others ( such as the enigmatically named “inktomi slurp” it’s just that the Gb is probably the most important and also the most resource consuming.
BA (Hons) Busking October 1, 2005
Posted by Andy Roberts in : learning , 6commentsYear Three of Ultraversity’s ground breaking online degree consists of an Action Research project devised by the student based on our own circumstances.
Working Title: Busking for improvement
Context:
It’s two months since my short term contract at Marsdon School wasn’t renewed, and there’s no sign of any fees coming in from my new IT consultancy business yet, so in desperate need of a bit of liquidity, the clock turns back thirty years: I picked up my old guitar and headed down town to work as an itenerant street singer (busker)
The Problem:
Busking in London is hard work so: how can I increase revenue and play less hours while still paying the bills?
Action:
I will make a series of busking expeditions each of a fixed time, trying out different techniques which I think might increase the takings. Based on an assumption that I perform best when I’m enjoying the songs I will also record how I’m feeling about the session every 10 minutes or 3 songs (qualitative) as well as counting the money, both in total and as sets of different denomination coins. (quantitative)
Participation:
In a further cycle of the enquiry I will employ the services of an assistant whose job is to hold the hat and collect the money (a bottler). The bottler will be invited to make suggestions for further cycles and we may make joint decisions on the fly about the playlist according to which songs appear to be going down well (emergent action). With adequate assistance it may be possible to collect video data as well. By convention, the takings will be split 50/50.
Audience:
The exhibition will be presented live to a populous but frequently changing audience of strangers, so I will need to have good mechanisms in place for collecting feedback data from a small self-selected volunteer sample of people who are willing to stop and leave comments.
Literature Review:
None, because “writing about music is like dancing about architecture” (Zappa, 1974)
Potential for further cycles.
Plenty. For example the location could be changed to Paris (via Eurostar) or the entire music element could be dropped and replaced with an empty vodka bottle and a dog on a piece of string to see if takings go up or down.
Ethics:
In consideration for audience sensitivities there will be no Simon and Garfunkel.
Tachnorati Tags:
ultraversity,
busking, actionresearch,
music
is an online professional who initiated DARnet 
