I spent last week running my computer 24/7 processing several gigabytes of data and eventually filled up a MySQL database table with 4Gb of data (I’ve still got 1Gb to load in!). So I’ve had to redesign the database scheme, branch out to flat file databases and now I’m trying to get the data out of the MySQL database into the flat files…
Except, it won’t export the data now. Every time I try and extract it out using a very very basic Perl script, it bombs out with “Out of memory (Needed XXXXX bytes)”. Despite the fact I’ve got 0.5Gb RAM and 4Gb of “swap space” currently configured.
I need to get this thing sorted ASAP – I want to be able to upload it this weekend (ready for a launch on the 1st of October) and I want to be able to use my PC again. Whilst it’s “number crunching” and processing and whatnot, my machine slows down to a crawl 🙁