How not to serve a web page:

home | blog | Teh Internet | guest blog |rants | political | projects | Gwen and Liam | Citadel patched | Tools | Scouts

- Careful Chrome users, this search box might be "Not secure"


P.S. Thanks, your crawlers are allowing me to learn new things.

O.K. update time. Looks like you won round 1.

I think it is cheating to use 16 bots at a time on one website.
And you should not be so predictable - every 6 hours seems to be your crawler interval.