Press "Enter" to skip to content

Richy's Random Ramblings

Linux: Handy Varnish commands

November 29, 2009
1 Comment

If you are running the Varnish reverse proxy cache system on your website (yep, this is a techy post!), you might find the following command line tools useful. Varnish is a very powerful and useful cache tool which sits in front of your website and helps reduce the load on Apache/PHP – but there’s very little information about how to use it available. Hopefully this is the first of many posts about perfecting a varnish configuration. But first, let’s get an idea of some of the information that can be reported from varnish:

varnishlog
See what Varnish is currently processing.

varnishtop -i RxHeader -I \^Referer
Show the referer (sic) header for requests.

varnishtop -b -i TxURL
Shows requests made to the backend (-b) where the line matches (-i) the transmit URL (TxURL). Basically, shows you what is being passed to the backend and isn’t being cached. It will list all requests going to a backend, grouped by URL and sorted by a decaying average of frequency. Basically the number on the left should be single-digit and preferably all 1s or less (a higher number means the backend request is taking place frequently). [Technically, you don’t even need the “-b” as the TxURL is only set when making requests to the backend anyway]

varnishhist
Shows a histogram chart of the last 1,000 requests (by default) to the Varnish proxy showing “|” as a “hit” on the cache and “#” as a miss. The more “|” to the left of the chart, the better. The scale on the bottom is in seconds with 1e0 being “1” second and 1e-6 being 0.000001seconds (1e-1 being 0.1seconds). The vertical scale is shown in the top left hand corner.

varnishstat
Shows various information about varnish – I’m sure I’ll figure out what they mean in time…

Use varnishreload
From http://kristian.blog.linpro.no/2009/02/18/easy-reloading-of-varnish-vcl/ , this is a very handy script you can use to reload varnish’s configuration without having to restart the proxy server:
#!/bin/bash
# Reload a varnish config
# Author: Kristian Lyngstol
FILE="/etc/varnish/default.vcl"
# Hostname and management port
# (defined in /etc/default/varnish or on startup)
HOSTPORT="localhost:6082"
NOW=`date +%s`
error()
{
echo 1>&2 "Failed to reload $FILE."
exit 1
}
varnishadm -T $HOSTPORT vcl.load reload$NOW $FILE || error
varnishadm -T $HOSTPORT vcl.use reload$NOW || error
echo Current configs:
varnishadm -T $HOSTPORT vcl.list

Show what caused recent 503 errors
varnishlog -d -c -o TxStatus 503

Clear stale cache by host name
purge req.http.host ~ example.com

See what status codes are being commonly hit by your users (ideally lots of 200 and few 5xxx)
varnishtop -i TxStatus
See also: http://www.slideshare.net/schoefmax/caching-with-varnish-1642989,
http://blogs.osuosl.org/gchaix/2009/10/12/pressflow-varnish-and-caching/ and http://letsgetdugg.com/2009/12/04/random-varnish-tips-n-tricks/

PHP: self and clone

October 21, 2009
Comments closed

I recently had to make a PHP class (let’s call it “class_A”) which returned an array of itself (to return multiple search results). All well and good and I used “$object=new self();” to do so like this:


class class_A {
public $setting;

public function doSomething() {
$toReturn=Array();
for ($i=1;$i<100;$i++) { $object=new self(); $object->setting=$i;
Array_push($toReturn,$object);
return $object;
}
}
Calling $class_A->doSomething() then returns 100 “class_A” objects.

However, I then needed to extend the PHP class (let’s call this class_B – declared with “class class_B extends class_A”) to take into account additional settings and I fully expected $object to be a new copy of class_B (as class B is technically the class being run).

Nope, it doesn’t work like that. I ended up with $object being class_A . I had to modify the code to change:
$object=new self();
to:
$object=clone $this;
which then allowed $class_B->doSomething(); to return 100 class_B objects.

Annoying, but something to remember!

Bouncing Emails – Some Stats

September 25, 2009
3 Comments

Well, yesterday at work we launched Manga High’s Educational Maths Games and we were featured on Radio 4’s “Today” show (6.6million listeners), the Guardian newspaper (0.35million paper readers) and Guardian website (25million uniques per month) and Channel 4’s “News at Noon” (around 1million viewers) and apart from a 3.5hour “outage” between 7.30 (Today show’s broadcast time) and 11am when the site was under terrific load, things went well and we had a very very high number of registrations and visitors.

However, we did have a number of incorrect email addresses entered into the system which caused bouncebacks the the stats were quite interesting:
* We had 1 bounce back from Yahoo
* We had 6 bounce backs from Hotmail
* We had 1 bounce back from Gmail (disabled account)
* We had 3 other bounce back – each from a different domain name
* We had 5 people who entered an incorrect domain name (which didn’t even look like Yahoo/Hotmails/Gmails domains).

I wonder what this indicates – that you are more likely to get people who cannot enter their email address correctly from Hotmail?….

Bad Ideas – 3 in 3 days!

September 17, 2009
Comments closed

My first bad idea was on Tuesday when we all (at work) had a glass of wine to celebrate moving to the new office. Not such a bad idea – until after I took my sleeping pills (yep, I’m having trouble sleeping and had forgotten I was taking them when I had the wine). Sleeping pills + alcohol is a very very bad idea – my partner said that I was “vibrating” during the night, couldn’t stand up straight and I was disorientated most of Wednesday.

My second bad idea was leaving the pub for the celebrations when I did (after the glass of wine, I was on orange juice). It was absolutely pouring down with rain for the 5 minutes or so it took me to get to the tube, and 30 minutes later when I got home I was still drenched.

My third, and so far, final bad idea was this morning. I thought I’d save myself a walk from Euston Square tube station to work by changing tube lines and getting off at Goodge Street station. Silly bad idea. I had to change at Kings Cross station, up a flight of steps across and down to the southern-bound Northern Line – where I had to change again at Euston for the southern-bound Northern Line (as it branches at Euston) which involved me going up 2 escalators, across, down, across down another escalator to end up at the opposite end of the same platform. Up 2 escalators again, across and down 2 got me to the correct southern Northern line. I then exited at Goodge Street station and, seeing the lifts were very crowded, decided to walk up the steps. 139 steps later, I was even more out of breath and sweating then I was. Lovely!

Anyway, I am beginning to get a little bit of free time back after work so I should be able to resume blogging and twittering in a few days! Woot!

Techy: Find and replace on Linux excluding SVN folders

September 7, 2009
Comments closed

Just another snippet aid to memory, this find and replace (using perl) uses Linux’s find’s “prune” syntax to “prune” the path before it to avoid doing and search and replace on .svn folders:

find -path '*/.svn' -prune -o -type f -exec /usr/bin/perl -pi -e "s/ORIGINAL/NEW/g" {} \;