How not to serve a web page:

home | blog | Terrible people and places | Covid-19 links | Teh Internet | guest blog |rants | placeholder | political | projects | Gwen and Liam | Citadel patched | Tools | Scouts


P.S. Thanks, your crawlers are allowing me to learn new things.

O.K. update time. Looks like you won round 1.

I think it is cheating to use 16 bots at a time on one website.
And you should not be so predictable - every 6 hours seems to be your crawler interval.