(?:Dis)?Organi(?:z|s)ed Chaos

So, it's the day after the afternoon before. In that time, we upgraded our corporate forums, suffered a DDoS attack, send out 12,000 emails, fixed 50 odd bugs and even found time to write in a legacy mode to enable the blog and gallery to work without modification.

The upgrade to 2.1 seems to have gone fairly well. It was a bumpy ride for the first few hours and the odd (hundred) bugs here and there did their bit to add to the overall chaos. Ever wondered what it's like to upgrade a relatively busy forum to a largely untested release? It goes something like this:

1:00pm: Turn company forums offline. Add in a redirect to a basic index.html file.
1:30pm: New file uploads complete
1:50pm: MySQL updates done surprisingly quickly. Note to self: MySQL 4 is so much better than MySQL 3.
2:00pm: Hack index.php to allow access for myself and the US office IP address.
2:10pm: Various staff members AIM me with bugs they've found.
2:14pm: It's apparently turned into a competition.
2:22pm: Fixed the first round of bugs (parse errors, the biggies)
2:34pm: AIM is lit up like a christmas tree
2:38pm: Fixed some more bugs. Start roughing out an announcement
2:43pm: AIM is lit up like the Empire State Building
2:44pm: Few more bugs are fixed.
3:00pm: Finished announcement. Load Safari to post it (saves logging out of Mozilla)
3:01pm: Notice the standard editor doesn't work at all with Safari.
3:20pm: It does now.
3:22pm: MySQL error when posting from Safari
3:30pm: Get a message from CTerry via Stewart 'Stop slacking'
3:31pm: Motivated, I submit the announcement again
3:32pm: Remove IP address filter and turn board back online
3:33pm: Charles starts sending emails to our subscribers
3:45pm: Can't load a forum. This can't be good. ssh [email protected]
3:50pm: MySQL locks up, 156 locked processes. Lindy?
3:51pm: MySQL restarted.
4:10pm: Ah, that topic table index? Yeah. I'll go add that now.
4:15pm: MySQL locks up again.
4:20pm: Oh yes, LOW PRIORITY was removed from the topic counter update
4:25pm: Server restarted
4:26pm: Things are much improved. Nearly 400 users active and load @ 0.3 per CPU
4:30pm: Initial feedback comes rolling in. Bug reports are posted everywhere.
4:30pm - 6:00pm: Crazy bug fixing period.

Or something like that, anyway.

I've spent the better part of today bug fixing and fiddling with bits of javascript that didn't work on Opera. Overall feedback has been good. Apparently some people have taken offence to the topic rating images and the rich text editor buttons. I was thinking that the blue posting form would have caused the most upset.