Month: September 2003

I spent last week running my computer 24/7 processing several gigabytes of data and eventually filled up a MySQL database table with 4Gb of data (I’ve still got 1Gb to load in!). So I’ve had to redesign the database scheme, branch out to flat file databases and now I’m trying to get the data out of the MySQL database into the flat files…

Except, it won’t export the data now. Every time I try and extract it out using a very very basic Perl script, it bombs out with “Out of memory (Needed XXXXX bytes)”. Despite the fact I’ve got 0.5Gb RAM and 4Gb of “swap space” currently configured.

I need to get this thing sorted ASAP – I want to be able to upload it this weekend (ready for a launch on the 1st of October) and I want to be able to use my PC again. Whilst it’s “number crunching” and processing and whatnot, my machine slows down to a crawl 🙁

Oh – if anybody is interested, there’s a job vacancy going around: contact me for more details if you’re interested (just leave a comment with your email address and I’ll forward further details):
“Rough job spec” follows:

Looking for a unique job position within Leicester? (UK)
Do you have experience or want experience with *any* or all of the following

* Search engine optimisation/placement – Our customers are looking to us to get their sites in the top ten in the major search engines such as Google, Altavista, Yahoo etc. Have you got any experience in this field or want to learn the secrets of the trade?

* Customer support – have you had a customer facing role where you interact with new or existing customers over the telephone and email?

* Server administration: We’ve got a number of Linux based webservers running Apache, Cpanel, Ensim, Ensim Pro and a few other software suites – have you had any experience of these from either administration or user end? How about Microsoft FrontPage, FTP or anything like that? We’re not necessarily looking for “l33t” people, but if you can tell a web site address from an email address it’s a start!

It’s a challenging role working in a small, but growing and dedicated team, with a customer base now in its thousands! Can you “work on you own” yet still interact in a team and do you like challenges without too much “set routine”?

It’s gone 7pm and I’m still at work and look to be here for another hour afterwards finishing a recruitment site (I was meant to be able to pass the design onto a coworker, but unfortently they didn’t “meet requirements” and hence I’m having to do all of the work myself). And once I get back home, I’ve got to give the DNS system a really good kicking as, by looking at things, my main website is still showing the “Under maintenance” page (how it can do that with Apache down I’m not sure: attempt to stop Apache comes up with “Apache not running” attempting to start it comes up with “segmentation fault”): and checking on DNS Report shows there’s some really strange things happening with my DNS at the moment.

Groan. Can’t I have an “easy week” for once?

My London web server (yep, I’ve got two dedicated web servers at the moment) has just had a minor crash and Apache has failed. Removing and reinstalling and configuring it still pulls up faults, but luckily I’m on the brink of decommissioning that 3year old server anyway – however, it wasn’t scheduled to happen for another fortnight at least to give me time to write all the scripts that run on it (it’s complete CGI driven).

So I’ve just had to copy all the files across to this server and try and juggle everything I can – I think it’s all worked as well as can be expected (still got a few problems with the scripting but I’ve patched around it for now), but I won’t be able to check if the DNS has properly updated for another 12 hours or so. The thing is, I’m not sure if I’ve got the configuration “spot on” – the old server was a totally customised DNS system and now it’s automated: but it’s still got to interact with my external slave nameservers (all 12 of them!). Fingers crossed – but if my blog disappears for a day, you’ll know what I screwed up (hopefully all my other sites will shortly update). It doesn’t really help that I’m “between registry’s” at the moment 🙁

Nooo!!!! 200,000 MySQL inserts comprising of over 2.16Gb of data won’t work because there’s a single comma missing in the SQL insert statement in every single row… Nooo! I just hope I can rescue the data with a Perl regexp or else I’ll have to wipe out the last 3.5days of work and processing (that’s 24/7 processing by the way: in the last ten days, I’ve downloaded over 5.3Gb of data and already made over 20,190,979 MySQL inserts – and I’ve still got all the above to do: working on project DBC is a lot harder than first expected!)