December 12th, 2004

Scott

Ah, yes... The perks of being a network administrator

The main file server blew up yesterday (Saturday) morning.  Fixed it with a reboot, but the RAID controller said it had a bad disk.

Called Dell, and they wanted me to do low-level (ie. take the system down) verification, which is reasonable, before they send me another disk.

Of course, it would be too much to ask for the user community to actually take the (expletive deleted) weekend off - we had people screaming at 7:50am about the server being down...  There's only 200-some people left, you'd think they might want to spend time with their families.

Anyway, decided to run a backup against the degraded volume before I played with it, and the backup took until 8:30pm - BUT the CRC verification on the tape took until midnight.

So, now I get to spend the night in the 65 degree computer room babysitting disk verifications and a RAID rebuild.  The first disk said it had *ONE* error, and when I rebooted Windows it says the volume is corrupt and I need to run CHKDSK.  Now, this is WHY I'm running RAID on the (there's that expletive again) thing - to keep that from happening.  So now I'm expecting that one of the other disks blew chunks (of course, the RAID controller could say something other than "all the disks are fine, really!"), and when the controller tried to fail off of that disk it hit the bad sector on the other disk - or maybe not.  Maybe it's all just a REALLY, REALLY BAD DREAM.

At least I've got the backup - hope I didn't have bozos in here playing with the data all night while the tape was getting verified.

I'm just waiting for the server to say "I can't do that, Dave"...
  • Current Music
    This Love - Maroon 5
Scott and Cindy

The day is OVER (ok, at least YESTERDAY is, anyway...)

I finally get to go to sleep.  Yay.  Only 36ish hours this time.

It finally turned out to be the RAID controller being nuts.  None of us (technical support or the technician or myself) had ever seen one be quite so squirrely before, but replacing the RAID controller parts (memory and the key) looks to have fixed the problem.

Luckily, my boss decided to come in and watch the server to make sure it rebuilds so I can get some sleep before my 1pm final tomorrow (which, of course, I haven't studied for because my brain is DEAD, Jim).  I'm hoping that we won't have any problems, and I'll get to study at work tomorrow (I'll have to go in to make sure that we don't have any problems, but hopefully I won't have to DO anything...).

Then, after class (and after Ryan gets out of school), we're going to take a grave pillow down to the cemetery for Christopher (a grave blanket really wouldn't work - he has a very small grave).  Cindy went and had our favorite flower shop make it just for him, and it looks very nice.  We were going to take it down this weekend, but that got shot to heck with the server problems.  But, with the 50-60 mile an hour winds we're having, maybe it was best to wait until tomorrow...
  • Current Mood
    exhausted exhausted