#user-agent
2 one-liners with this tag.
All one-liners tagged user-agent
Fetch a page as Googlebot and compare with a normal request
$for ua in 'Mozilla/5.0 (X11; Linux x86_64)' 'Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)'; do curl -s -o /dev/null -A "$ua" -w "%{http_code} %{size_download} Bytes $ua\n" https://example.com/; done
Find the most common user agents and bots in the access log
#awk -F'"' '{print $6}' /var/www/vhosts/system/example.com/logs/access_ssl_log | sort | uniq -c | sort -rn | head -20