Aaargghhhh! Our main internet connection has gone down – we alerted our main ISP at 11.50am (first one to do so: shows how much on the ball I am, plus I had conduct all the necessary tests to be able to tell them what the problem is ). It’s now 3.10pm and we’ve had to resort to dialup. Correction – I’ve had to resort to dialup to our backup ISP: therefore all urgent internet related stuff is having to pass thru me. Problem of being system administrator

Whilst we’ve got the remote backup mail servers+name servers hosted on another backbone (as per good internet setups ), we decided that since our ISP had been so good so far, we didn’t need full broadband access from another ISP (saves money too). Now I’ve got the whole business getting it’s connectivity thru one 56k dialup modem attached to a fax line: not good, but hopefully it’s only temporary. I hope it is, as the fax machine don’t sound happy sharing the line (I can hear the modem thru the fax’s speaker).

It’s not the best backup solution in existence, but it works, it does its job (ie no data loss, still can get essential emails etc), and is cost effective. Ok, if it’s still down at 4pm, we may have to do something major but fingers crossed….

[edit]
3.45pm and normality has been resumed. Our ISPs engineers are still investigating the issue to try and ensure it doesn’t happen again.

After a bit of prompting via this slashdot story (also on The Register) about the UNIX DNS server BIND having a vulnerability, I decided to upgrade from BIND 8.3.3 to BIND 9. Apart from having to re-build the named.conf configuration file, it needed very little work to get running again. I also took to opportunity to start using ACL (Access Control Lists) in the configuration file to make things easier to read and to cut down and repetition.

Yesterday (8th November), my main server in London went ‘down’ around 6.30am. Unfortunately, I didn’t notice until around 5.30pm (on a Friday night!) and instantly sent my hosting provider an email via their web-feedback form to ask them to re-boot my server to try and bring it back online. I send them a proper email at 6.38pm as well (just in case). I can’t sleep so at 9am, I try calling them on the telephone. No response. At 10am, I send an email CC’ing to their sales address (and, this time, get an automatic acknowledgement). Around midday, I get a very apologetic phone call saying that they had moved all their dedicated servers to another part of their hosting facility, but somehow mine had got lost behind.
» Read more: Techy: Server downtime

It seems that the DNS changes I put in place yesterday seem to have “taken” correctly. But unfortunately, my home PC seems to have cached the DNS records so I can’t check to the degree I want to. Remote checks seem to indicate all is well, but I wanted to do a DNS ‘dig’ from my home machine (as I can’t do it from anywhere else because I’ve only allowed a set group of IP address that sort of access to the server). Looks like I’ll have to wait until I’ve rebooted to find out (I was actually thinking of giving my machine a few hours downtime tonight so I can get some data off my old machine – I’ve only got the one monitor you see, plus I want to give my main machine a few hours ‘rest’: it’s normally on 24/7!).

I’ve just requested that Network Solutions change the DNS servers of one of my domains to point to Name Servers I have direct control of (previously they were pointing at my web host’s NS’s, but I wanted a bit more control over the entries – making sub domains etc etc.

Thing is, although I’ve tested the DNS setup (and had it running on 4 other domain names for over a month), I’m still slightly nervous about this switch over. Mainly because if the DNS goes tits-up (which it shouldn’t do), then my website and main email address are off-line for 24 hours until I switch the DNS servers back. Well, fingers crossed – nothings gone wrong so far in the last few hours (the change over happened around 3 hours ago), I see no reason how it could go wrong, but… Just call me “Richy – the paranoid humanoid”. Heh.