Not good news

Unfortunately the upgrade has not gone to plan. The servers upgraded fine, the compact went without a hitch but when the admin team tried to restart both servers in the cluster crashed with a JVM error message.

Some very quick investigation work has narrowed the problem down to one of our jar files which is used by servlets but I do not yet understand why this file should cause a problem, it is fine on our dev and both tesst servers.

So queue an emergency phone conference at 9am tomorrow. I have to be in at 7 to test a workaround which I have proposed but even if it works I can't see the upgrade continuing. Then we will be into emergency rollback mode which is not difficult but is time consuming simply due to the size of the files involved. And then we'll have to try again next weekend once we have spoken to Lotus Support during the week. Which ties in rather nicely with an interesting discussion currently happening on Ed's blog.

I was meant to be going out tonight to see Richard Herring but I'm not sure I can face it now. I'll have some dinner and then decide.