I just discovered that I've been receiving a message from "mdadm monitoring" every morning at about 8:00 am since July 2012, but these messages were delivered to root and then went to "nobody" because I never set a root email address in /etc/aliases. Anyway, it looks like these messages are all the same and they look like this: Date: Wed, 2 Sep 2015 07:55:39 From: mdadm monitoring <root at taxa1> To: root at taxa1 Subject: SparesMissing event on /dev/md/1:taxa1 This is an automatically generated mail message from mdadm running on taxa1 A SparesMissing event had been detected on md device /dev/md/1. Faithfully yours, etc. P.S. The /proc/mdstat file currently contains the following: Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10] md1 : active raid1 sda3[0] sdb3[1] 2896392511 blocks super 1.2 [2/2] [UU] md0 : active raid1 sda2[0] sdb2[1] 33553336 blocks super 1.2 [2/2] [UU] unused devices: <none> ---------end of message on previous line------------- Maybe this isn't telling me anything I didn't already know -- the raid doesn't boot normally and probably only one of the two drives has grub properly installed. Mike On Mon, 31 Aug 2015, Mike Miller wrote: > Despite all the help I had, way back in October 2012, I still never fixed > that RAID1. At this point I think I know what went wrong. I would like to > try to fix it so that I can upgrade the Ubuntu LTS version -- I'm still using > 12.04.5 LTS. > > See the output below from df, parted -l, /proc/mdstat and /etc/fstab. There > is a /boot directory, but that is in the / partition. The /boot partition is > not mounted and it isn't in the RAID1. > > I have been doing software updates, and kernels have been installed, but the > only one I can boot from is the one I installed originally. I assume that > since the /boot partition isn't mounted, the dozen or so kernels I see are in > the /boot directory in the / partition, and there are 341 MB of files in > there now. > > I'm hoping there is a way to get the appropriate files into the /boot > partition, get that mounted, get rid of the /boot directory in the / > partition and have the /boot partition mirrored on the two drives. > > Is there hope? Do any of you know how I should proceed? > > Mike > > > $ df -HT > Filesystem Type Size Used Avail Use% Mounted on > /dev/md1 ext4 3.0T 296G 2.6T 11% / > udev devtmpfs 8.4G 13k 8.4G 1% /dev > tmpfs tmpfs 3.4G 906k 3.4G 1% /run > none tmpfs 5.3M 0 5.3M 0% /run/lock > none tmpfs 8.4G 52M 8.4G 1% /run/shm > > > $ sudo parted -l > > Model: ATA ST3000DM001-9YN1 (scsi) > Disk /dev/sda: 3001GB > Sector size (logical/physical): 512B/4096B > Partition Table: gpt > > Number Start End Size File system Name Flags > 1 10.5MB 220MB 210MB /boot bios_grub > 2 220MB 34.6GB 34.4GB swap raid > 3 34.6GB 3000GB 2966GB / raid > > > Model: ATA ST3000DM001-9YN1 (scsi) > Disk /dev/sdb: 3001GB > Sector size (logical/physical): 512B/4096B > Partition Table: gpt > > Number Start End Size File system Name Flags > 1 10.5MB 220MB 210MB /boot bios_grub > 2 220MB 34.6GB 34.4GB swap raid > 3 34.6GB 3000GB 2966GB / raid > > > Model: Linux Software RAID Array (md) > Disk /dev/md0: 34.4GB > Sector size (logical/physical): 512B/4096B > Partition Table: loop > > Number Start End Size File system Flags > 1 0.00B 34.4GB 34.4GB linux-swap(v1) > > > Model: Linux Software RAID Array (md) > Disk /dev/md1: 2966GB > Sector size (logical/physical): 512B/4096B > Partition Table: loop > > Number Start End Size File system Flags > 1 0.00B 2966GB 2966GB ext4 > > > > $ cat /proc/mdstat > Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] > [raid10] > md1 : active raid1 sda3[0] sdb3[1] > 2896392511 blocks super 1.2 [2/2] [UU] > > md0 : active raid1 sda2[0] sdb2[1] > 33553336 blocks super 1.2 [2/2] [UU] > > unused devices: <none> > > > > # /etc/fstab: static file system information. > # > # Use 'blkid' to print the universally unique identifier for a > # device; this may be used with UUID= as a more robust way to name devices > # that works even if disks are added and removed. See fstab(5). > # > # <file system> <mount point> <type> <options> <dump> <pass> > proc /proc proc nodev,noexec,nosuid 0 0 > # / was on /dev/md1 during installation > UUID=f8a6d38d-3dd0-465e-b622-1cffdde52c1b / ext4 > errors=remount-ro 0 1 > # swap was on /dev/md0 during installation > UUID=f8be4924-e4a8-20ba-f284-3a90581a5322 none swap sw > 0 0 > > > > On Fri, 5 Oct 2012, Mike Miller wrote: > >> On Fri, 5 Oct 2012, ron at ron-l-j.com wrote: >> >>> The first thing I would try is booting into a shell and running fsck on >>> your boot disk. >>> Reboot the machine and press e in the grub screen for edit. >>> Go to the kernel line and press e to edit that line. >>> Then add to the kernel line >>> >>> init=/bin/bash >>> >>> when you boot you go straight to the shell. >>> then do a file system check on your boot drive >>> fsck -t ext3 /dev/sdax >>> t is for file system type in this case ext3 >>> and the x in sdax is your partition number. >>> sda is for sata drives, if you have an ide drive use hdax >>> >>> Most of the time its a file system error and was very common when I was >>> doing raid arrays in my advanced linux classes. Bad super block is also >>> common. >>> If you are still having troubles run fsck from a live cd/usb. But you will >>> have to run the cd in live mode, mount the offending drive, and chroot >>> into the drive. >>> As a last resort you can update-grub from the live cd after you have >>> chroot'ed into you installed environment. >>> Let me know what you encounter. >> >> >> Thanks, Ron. (I'm cc'ing you in case you are doing digests only and want >> to see this before the digest comes in.) >> >> Did you see that I was able to boot up to the window manager login prompt >> using an earlier version of the kernel? Can I just take it from there? >> >> My impression right now is that the problem was caused by my failure to >> reboot for weeks after installing some packages that required rebooting. >> Then I installed 400 more packages before rebooting. Is that possibly the >> cause of my troubles? Could it be that I just need to fix the packages and >> reboot? >> >> The other issue is that I have a RAID1, so mirrored drives, and I think >> that means I don't want to fsck them one at a time. df shows this: >> >> $ sudo df -HT >> Filesystem Type Size Used Avail Use% Mounted on >> /dev/md1 ext4 3.0T 220G 2.6T 8% / >> udev devtmpfs 8.4G 13k 8.4G 1% /dev >> tmpfs tmpfs 3.4G 832k 3.4G 1% /run >> none tmpfs 5.3M 0 5.3M 0% /run/lock >> none tmpfs 8.4G 148k 8.4G 1% /run/shm >> >> Does that mean I would fsck /dev/md1? >> >> Thanks again. I really appreciate your taking the time to reply. >> >> (One thing I've learned -- I should run package-manager daily and try to >> reboot soon when it is required. I often have a lot of stuff running that >> I don't want to kill, so reboots are a hassle.) >> >> Mike >> >