Cannot mount root on grub

marianogedisman

marianogedisman

New Member
Joined
Apr 21, 2021
Messages
26
Reaction score
5
Credits
241
Hello! I installed Debian 12 on a VMware VM.

It was working fine until I rebooted it and then I was greeted with grub.

Based on the current volume info:

Code: 
grub> Is
(proc) (memdisk) (lvm/debian--vg-home) (lvm/debian--vg-tmp)
(lvm/debian--vg-swap_1) (lvm/debian--vg-var) (lvm/debian--vg-root) (hd0)
(hd0,gpt3) (hd0,gpt2) (hd0,gpt1) (cd0) (cd0, apple2) (cd0,apple1) (cd0,msdos2)

Code: 
grub> Is (hd0,gpt2)/
lost+found/ efi/ config-6.1.0-22-amd64 vmlinuz-6.1.0-22-amd64 grub/ System.map-6.1.0-22-amd64 initrd.img-6.1.0-22-amd64
grub> Is (lvm/debian--vg-root) /
Possible files are:
lost+found/ boot/home/ tmp/ var/ etc/ media/ vmlinuz.old usr/ lib lib64 sbin bin dev/proc/ root/ run/sys/ mnt/srv/opt/initrd.img.old vmlinuz initrd.img

I tried the following commands:

Code: 
grub> linux (hd0,gpt2)/vmlinuz-6.10.0-22-amd64
grub> initrd (hd0,gpt2)/initrd.img-6.10.0-22-amd64
grub> set root=(lvm/debian--vg-root)
grub> boot


Which gives me a booting Linux but stuck at:

Code: 
Begin: Running /scripts/local-block … done
Begin: Running /scripts/local-block … done
Begin: Running /scripts/local-block … done
Begin: Running /scripts/local-block … done
Begin: Running /scripts/local-block … done
Begin: Running /scripts/local-block … done
Begin: Running /scripts/local-block … done

And then greeted by (initramfs) where I have an empty fstab file, thus I can't mount any device (even by readding them), I tried mount -a but still no luck, getting all sorts of kernel panics.

Of course, I could redo the VM but my goal is to learn from this situation in case I run into this issue in Production env. I've googled to no avail only to find all the steps above but still unable to mount root.

Any help or pointers to documentation would be greatly appreciated!
 


marianogedisman said:
grub> linux (hd0,gpt2)/vmlinuz-6.10.0-22-amd64
grub> initrd (hd0,gpt2)/initrd.img-6.10.0-22-amd64
Click to expand...

Shouldn't be the opposite? you first load initrd and then kernel? so:

Bash: 
grub> initrd (hd0,gpt2)/initrd.img-6.10.0-22-amd64
grub> linux (hd0,gpt2)/vmlinuz-6.10.0-22-amd64
etc...
 
Hello @CaffeineAddict and @GatorsFan ! Thanks for your input. I've tried that way and I get this error:

1723601216394.png
 
osprey said:
Perhaps try this order:
Code: 
grub> set ...
grub> linux ...
grub> initrd ...
grub> boot ...
Click to expand...
Hello! I tried that, but still the same. I even found this guide: https://www.linuxfoundation.org/blo...n-how-to-rescue-a-non-booting-grub-2-on-linux and quotes the same order as you mention:

Code: 
grub> set root=(lvm/debian--vg-root)
grub> linux (hd0,gpt2)/vmlinuz-6.1.0-22-amd64 root=/dev/sda1
grub> initrd (hd0,gpt2)/initrd.img-6.1.0-22-amd64
grub> boot
But even following this order I still end up in (initramfs) after Begin: Running /scripts/local-block . I guess it's because my root and kernel are in different partitions?

1723606172441.png



Once I'm in in (initramfs) and I run blkid I get this:

1723606570236.png


However, if I try linux (hd0,gpt2)/vmlinuz-6.1.0-22-amd64 root=/dev/mapper/debian--vg-root I get back to Kernel panic. I am sure it's because root must be in the same partition than linux or something like that?


One of the things I wanted to try was to add one of those partitions to fstab and see if that works but not sure how to do it in this case.
 
Last edited:
Also tried adding sda2 to /etc/fstab, it mounted just fine but when I exit from initramfs I get a kernel panic as well. I'm at loss.
 
@marianogedisman wrote:

<snip>
Hello! I tried that, but still the same. I even found this guide: https://www.linuxfoundation.org/blo...n-how-to-rescue-a-non-booting-grub-2-on-linux and quotes the same order as you mention:
<snip>
Click to expand...

I understand that you may wish to resolve the problem you are having with the current virtual machine installation. Without wishing to distract you from your task as you've presented it, I wondered whether in my own case, would I do it that way.

Here are some observations which may or may not be of use to you.

The installation you are running is a virtual machine. Virtual machines are so amenable to being discarded and recreated, so if one wants to check out an installation, it's not usually so much a problem to delete and start over.

Is there a special reason for running LVM partitioning rather than standard partitions? Are you intending to "play" with partitioning? LVM partitioning is useful on a server where partition sizes may need to vary over time and be manipulated as demands change, e.g. if new disks are added to an array or unexpected installations need to be made. For personal computing, LVM is usually an unnecessary extra layer of software that is irrelevant unless there is some special reason to use it. Some distros may use LVM as a default partitioning system but that's generally because they try to cover all grounds rather than cater to individual needs.

As you appear to be aware, normally the kernel, vmlinuz, and the initramfs image are in the /boot directory, so that one has a root directory, within which is a boot directory, within which are the kernel and initramfs image.

Here is a normal file structure for /boot where the kernel and initramfs are usually found:
Code: 
[tom@min ~]$ tree -L 1 /boot
/boot
├── config-6.10.3-amd64
├── efi
├── grub
├── initrd.img-6.10.3-amd64
├── System.map-6.10.3-amd64
├── vmlinuz-6.10.3-amd64
<snip>

Here is a file structure for / where there are links to the kernel and initramfs image pointing to the /boot directory:
Code: 
[tom@min ~]$ tree -L 1 /
/
├── bin -> usr/bin
├── boot
├── dev
├── etc
├── home
├── initrd.img -> boot/initrd.img-6.10.3-amd64
├── lib -> usr/lib
├── lib64 -> usr/lib64
├── lost+found
<snip>
├── vmlinuz -> boot/vmlinuz-6.10.3-amd64

Not all distros include the links of the kernel and initramfs image from / to /boot, but if they exist, they can be used in the grub> prompt.

In a virtual machine, often an installation is a directory structure where a root directory includes a boot directory under it, all on the one standard partition, with other partitions e.g. for efi or swap only if necessary.

In the output in post #7 the kernel cannot find a root filesystem and returns "No such device". It looks like it's the file structure has failed the kernel, rather than the kernel or grub failing because of any disorder of their own. If that's the case, perhaps attention to the file structure may be useful, for example, creating partitions and their contents that are known to work :). YMMV.
 
Last edited:
marianogedisman said:
grub> set root=(lvm/debian--vg-root)
Click to expand...
I'm pretty sure problem is in debian--vg-root name which is what's reported by grub and also lsblk however the real LV name is probably different and you need to figure out what it is.

See my related solved problem below:

Also I think you should set /boot partition in grub to boot or maybe /boot/efi which contains boot loader, not root partition.
 
osprey said:
@marianogedisman wrote:



I understand that you may wish to resolve the problem you are having with the current virtual machine installation. Without wishing to distract you from your task as you've presented it, I wondered whether in my own case, would I do it that way.

Here are some observations which may or may not be of use to you.

The installation you are running is a virtual machine. Virtual machines are so amenable to being discarded and recreated, so if one wants to check out an installation, it's not usually so much a problem to delete and start over.

Is there a special reason for running LVM partitioning rather than standard partitions? Are you intending to "play" with partitioning? LVM partitioning is useful on a server where partition sizes may need to vary over time and be manipulated as demands change, e.g. if new disks are added to an array or unexpected installations need to be made. For personal computing, LVM is usually an unnecessary extra layer of software that is irrelevant unless there is some special reason to use it. Some distros may use LVM as a default partitioning system but that's generally because they try to cover all grounds rather than cater to individual needs.

As you appear to be aware, normally the kernel, vmlinuz, and the initramfs image are in the /boot directory, so that one has a root directory, within which is a boot directory, within which are the kernel and initramfs image.

Here is a normal file structure for /boot where the kernel and initramfs are usually found:
Code: 
[tom@min ~]$ tree -L 1 /boot
/boot
├── config-6.10.3-amd64
├── efi
├── grub
├── initrd.img-6.10.3-amd64
├── System.map-6.10.3-amd64
├── vmlinuz-6.10.3-amd64
<snip>

Here is a file structure for / where there are links to the kernel and initramfs image pointing to the /boot directory:
Code: 
[tom@min ~]$ tree -L 1 /
/
├── bin -> usr/bin
├── boot
├── dev
├── etc
├── home
├── initrd.img -> boot/initrd.img-6.10.3-amd64
├── lib -> usr/lib
├── lib64 -> usr/lib64
├── lost+found
<snip>
├── vmlinuz -> boot/vmlinuz-6.10.3-amd64

Not all distros include the links of the kernel and initramfs image from / to /boot, but if they exist, they can be used in the grub> prompt.

In a virtual machine, often an installation is a directory structure where a root directory includes a boot directory under it, all on the one standard partition, with other partitions e.g. for efi or swap only if necessary.

In the output in post #7 the kernel cannot find a root filesystem and returns "No such device". It looks like it's the file structure has failed the kernel, rather than the kernel or grub failing because of any disorder of their own. If that's the case, perhaps attention to the file structure may be useful, for example, creating partitions and their contents that are known to work :). YMMV.
Click to expand...

Hey @osprey ! Thanks again for the detailed response! I totally agree about the ephemeral nature of VMs, thus the reason why I prefer them on testing environments.
In this case was more a case of wanting to learn how to fix an issue I haven't experienced before!

Is there a special reason for running LVM partitioning rather than standard partitions? Are you intending to "play" with partitioning? LVM partitioning is useful on a server where partition sizes may need to vary over time and be manipulated as demands change
Click to expand...

Indeed! That's exactly the reason why I use LVM. This test environment is to be later rolled out on prod where partition sizes will vary and unexpected installations may happen (scalability as well) so you hit it right in the nail on that one!

Thanks a lot for the tree view of what it should look like, I will definitely jump into that VM and start comparing the info, let's see what I can take from all this invaluable info. Thanks again!
 
CaffeineAddict said:
I'm pretty sure problem is in debian--vg-root name which is what's reported by grub and also lsblk however the real LV name is probably different and you need to figure out what it is.

See my related solved problem below:

Also I think you should set /boot partition in grub to boot or maybe /boot/efi which contains boot loader, not root partition.
Click to expand...

Yep, I think you got quite a strong lead over there @CaffeineAddict.

Since all my "root" is in debian--vg-root but grub doesn't like the vmlinuz and init files in there and only accepts the vmlinuz-6.1.0-22-amd64 on (hd0,gpt2) , I tried copying vmlinuz-6.1.0-22-amd64 to debian--vg-root, but threw "Unexpected end of file error"

I'm thinking about doing the opposite and moving all the root partition contents to (hd0,gpt2).

I will definitely read your related solved problem link and try setting /boot as indicated. Thank you again!
 
Thanks everyone for taking the time to help me learning through this new process with Grub. I personally really appreciate your time and patience guys!
 
CaffeineAddict said:
See my related solved problem below:

Also I think you should set /boot partition in grub to boot or maybe /boot/efi which contains boot loader, not root partition.
Click to expand...
I also did check this out but lsblk and lvdisplay are not available in BusyBox ((initramfs)).


CaffeineAddict said:
I'm pretty sure problem is in debian--vg-root name which is what's reported by grub and also lsblk however the real LV name is probably different and you need to figure out what it is.

See my related solved problem below:

Also I think you should set /boot partition in grub to boot or maybe /boot/efi which contains boot loader, not root partition.
Click to expand...

Also, you mean set root partition? Or is there a way to set a boot partition? I've looked up all the set env vars and could not find anything about setting boot.

https://www.gnu.org/software/grub/manual/grub/grub.html#Environment




Btw @CaffeineAddict ! When I mount debian--vg-root I get this message:

1723680817627.png


Maybe that's the partition name I'm looking for?

1723680905483.png




UPDATE: Setting grub root as /dev/dm-0 did not work.
 
Last edited:
marianogedisman said:
UPDATE: Setting grub root as /dev/dm-0 did not work.
Click to expand...
I think one of the following should be booted:
(hd0) (hd0,gpt3) (hd0,gpt2) (hd0,gpt1)

use grub's ls to display more info about those partitions.

When you figure out which one is /boot partition (ex. likely (hd0,gpt1)) then do:

Bash: 
insmod part_gpt
insmod chain
insmod fat
set root="(hd0,gpt1)"
chainloader /EFI/debian/grubx64.efi

grubx64.efi should be the name of boot loader, but if you have secure boot enabled on that system then likely you need to boot shimx64.efi

Replace debian in the path above with your system NETBIOS host name.
 
CaffeineAddict said:
I think one of the following should be booted:
(hd0) (hd0,gpt3) (hd0,gpt2) (hd0,gpt1)

use grub's ls to display more info about those partitions.

When you figure out which one is /boot partition (ex. likely (hd0,gpt1)) then do:

Bash: 
insmod part_gpt
insmod chain
insmod fat
set root="(hd0,gpt1)"
chainloader /EFI/debian/grubx64.efi

grubx64.efi should be the name of boot loader, but if you have secure boot enabled on that system then likely you need to boot shimx64.efi

Replace debian in the path above with your system NETBIOS host name.
Click to expand...

Indeed it was (hd0,gpt1), however after setting all up as indicated, still getting this error:

1723688609451.png



Almost as if Grub would refuse to pull up the root partition.


Up to this point I think it's wise to call it quits, I don't want to waste any more of your time guys.

I have to reckon, I've learn a great deal out of you all! I really appreciate it, @CaffeineAddict and all the rest! Thanks a lot for giving me a great lesson on Grub, now I have a few tools to try and defend myself if some issues arise in production!
 
marianogedisman said:
Thanks a lot for giving me a great lesson on Grub, now I have a few tools to try and defend myself if some issues arise in production!
Click to expand...
Good luck! to master grub start here:

GNU GRUB - Documentation

Site for GNU GRUB
www.gnu.org

It's not user friendly documentation but it provides hints which you can google out to search for some examples.

marianogedisman said:
Almost as if Grub would refuse to pull up the root partition.
Click to expand...
Btw. try with set root="(hd0)", if not working try each one of them until one works.
 
You must log in or register to reply here.

Members online

Total: 426 (members: 2, guests: 424)

Latest posts

Top