#googlebot

3 one-liners with this tag.

Analyze Googlebot requests in the access log

#grep -i "googlebot" /var/www/vhosts/system/example.com/logs/access_ssl_log | awk '{print $7}' | sort | uniq -c | sort -rn | head -20

Fetch a page as Googlebot and compare with a normal request

$for ua in 'Mozilla/5.0 (X11; Linux x86_64)' 'Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)'; do curl -s -o /dev/null -A "$ua" -w "%{http_code} %{size_download} Bytes $ua\n" https://example.com/; done

harmlessSEO checks

Verify a real Googlebot via reverse DNS

$n=$(host 66.249.66.1 | awk '/pointer/ {print $NF}'); echo "PTR: $n"; host "$n"

harmlessSEO checks