Just a spot to remember Varnish tidbits.
See what is happening for requests (right now) via varnish:
varnishtop -i rxurl or for everything in ncsa / apache log format: varnishncsa If you want counts of who from where (assuming proxy setup): varnishtop -i TxHeader -I '^X-Forwarded-For:'
Or if you want response time histogram:
varnishhist
Reload the vcl (and not the cache):
service varnish reload
Remember to put the shared memory file on tmpfs:
echo "tmpfs /var/lib/varnish tmpfs nosuid,noatime,nodiratime,size=150M 0 0" >> /etc/fstab'
Default is 80 MB, don't think there is overhead, but keeping it at 150MB should be good for now if we can afford it.
varnishadmin for tuning on the fly. Save to the vcl to make it stick.
To verify Varnish is good to go, make sure to monitor uptime.
Varnish will syslog when the child dies, but the uptime parameter via a local check for Nagios (or your monitoring too of choice) is a better bet.
Since uptime is an up counter, you will need to think about the thresholds for critical and warning.
Logging the x-forwarded-for header
Will Jackson has a great article here:
http://willjackson.org/blog/configure-vanish-forward-client-ip-addresses-apache-logs
And as always, in case that link or site goes away, here are the tidbits:
Add the x-forwarded-for block to the varnish config vcl:
-----------
if (req.http.x-forwarded-for) {
set req.http.X-Forwarded-For = req.http.X-Forwarded-For + ", " + client.ip;
} else {
set req.http.X-Forwarded-For = client.ip;
}
-----------
Create a new custom log type:
echo 'LogFormat "%{X-Forwarded-For}i %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" varnishcombined' > /etc/apache2/conf.d/varnish-log
Change the CustomLog line in the sites you want to see the forwarded ip header from varnish to be the new varnishcombined log format.
For Apache 2.4:
Create:
/etc/apache2/conf-available/remoteip.conf
RemoteIPHeader X-Forwarded-For
RemoteIPInternalProxy ip.add.re.ss
RemoteIPInternalProxy 127.0.0.1
Remote IP Internal Proxy has less restrictions - i.e. will allow ipv4 RFC 1918 addresses - but feel free to use the more restrictive
RemoteIPTrustedProxy directive as needed. Use ngrep or similar to see what is being sent in the field for addresses to verify your needs.
Edit /etc/apache2/apache2.conf logging (replaced %h with %a) - old line is one with #
#LogFormat "%h %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"" combined
LogFormat "%a %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"" combined
a2enmod remoteip
a2enconf remoteip
apachectl configtest ( look for ok of course before proceeding with a restart )
systemctl restart apache2.service
Get the current running varnish config:
varnishadm vcl.list (list of configs).... blah varnishadm vcl.show "blah"
If you are getting 503 errors, you can try to capture logs with this to try to debug:
varnishlog -q 'RespStatus == 503' -g request Probably block the bad actor with iptables (or firewall-cmd rules) - if you use UFW, you are on your own: iptables -I INPUT -s off.en.der.ip -j DROP firewall-cmd --add-rich-rule='rule family="ipv4" source address="off.en.der.ip" reject'