Wednesday, January 25, 2017

Installing Linux (Ubuntu) on existing Windows NTFS partition

I need to install Ubuntu 16.04.1 on the computer, it should work on newer versions of Ubuntu too. But I am only allowed to write everything on an existing NTFS partition (i.e. The Windows Partition). I am not allowed to re-partition the NTFS partition, I am also not allowed to modify the partition table of the hard disk in any way, and even, I am not allowed to modify the MBR.

Seems Impossible? No.
I can write and modify any data in the NTFS partition, and this is more than enough.

The following serves a log for me, you can try this out if you want. It basically creates a RAW image file on the NTFS partition, and formatted it as EXT4. We then fire up the Ubuntu Live CD and mount the image as loopback file system, and hence, install Ubuntu on it. About the bootloader part, I used Grub2Win to add an entry for GRUB in the Windows 7 bootmgr BCD record. Grub2Win should work on EFI partitions too (Tested by the author of Grub2Win, it works.). After successfully booted into GRUB, it is just the matter of accessing the kernel and initrd on the image file, and pass the correct cmdline into the kernel.

Image Creation
  1. Boot up the Ubuntu Live CD, or use existing linux machine to do this.
    Please transfer the image back to the NTFS partition if you are using another linux machine.
  2. Mount the NTFS partition. Simply clicking the partition on the file manger would have mounted the partition.
  3. Create an ext4 image file on <ntfs_partition>/linux/linux.img:
    cd /media/ubuntu/<your_ntfs_partition>/
    mkdir linux
    cd linux/
    dd if=/dev/zero of=./linux.img bs=1M count=32768 # This will create a 32GB Image for the Ubuntu system
    mkfs.ext4 ./linux.img
    

Installing Ubuntu on the image
  1. We need to trick the Ubuntu installer our image file is a real partition.
    I have created a virtual block device on /dev/sdb (You should adjust this accordingly), and mounting our image onto it:
    sudo mknod /dev/sdb b 7 200 # Make sure you have picked up a device name that is not in use (i.e. I only got sda, so i used sdb)
    # Also pick a minor number (200 used in here) that is not in use by checking "ls -la /dev" for existing loopback device(s).
    # Now do losetup
    sudo losetup /dev/sdb ./linux.img
    
  2. Now execute the Ubuntu installer, and install Ubuntu on /dev/sdb. You may prompted with this:
  3. Make sure you have clicked No, as our image file is mounted and reside on /dev/sda, unmounting /dev/sda will unmount our image file.
  4. Press "Something else":
  5. Select /dev/sdb and press Change...
  6. Select Ext4, and modify the mount point to /:
  7. You should then have the following configurations:
  8. I am not creating any swap space, but you can do so by creating one more swap image with mkswap and mount it with the method mentioned above. Press Continue here.
  9. We have already formatted it before with mkfs.ext4 above, so there is nothing in that partition. Press Continue here.
  10. Again, we have already formatted it before with mkfs.ext4 above, there is no files in the partition which would interfere the installation. Press Continue here.
  11. Configure timezone, keyboard layout, name, computer name, account, etc. as usual.
  12. Installing the system:
  13. At the end of the installation, there is an error about GRUB installation. Just select Continue without a bootloader. as we will install our own bootloader (Grub2Win) later.
  14. Do not restart yet, press Continue Testing
  15. We need to get the file names of the kernel and initrd. Mount the image and list the files under /boot:
    sudo mkdir /media/ubuntu/rfs
    sudo mount -o loop /dev/sdb /media/ubuntu/rfs
    ls -al /media/ubuntu/rfs/boot
    
  16. Mark down the file names for the kernel and initial ramdisk. In my case, it's vmlinuz-4.4.0-31-generic for the kernel and initrd.img-4.4.0-31-generic for the initrd.

Installing and configuring the bootloader
  1. Download and install Grub2Win
  2. Launch Grub2Win, press Manage Boot Menu
  3. Press Add A New Entry
  4. Select ubuntu as OS Type and fill up the Title
  5. Select I Will Enter My Own Custom Configuration Code
  6. Press Edit Custom Code
  7. A notepad will then popped up. Enter the following contents into the notepad.
    Please modify the disk and partition number hd0, msdos1, and the kernel and initrd file name according to your system setup and the file names you marked down in the previous step.
    You should also modify the root parameter passed to the cmdline to the linux block device path of your NTFS partition.
        echo **********************************
        echo * Ubuntu IMG boot by HopkinsKong *
        echo **********************************
    
        echo Booting linux...
    
        set gfxpayload=1024x768
        loopback loop0 (hd0,1)/linux/linux.img
        set root=(loop0)
        linux /boot/vmlinuz-4.4.0-31-generic root=/dev/sda1 loop=/linux/linux.img rw verbose nosplash
        initrd /boot/initrd.img-4.4.0-31-generic
    
    
  8. After saving and closing the notepad, you should get the following in the Grub2Win screen. Press the Apply button and the OK button to apply our new boot configuration.
  9. I prefer my Ubuntu boot entry have a higher order in the menu. So I pressed the up arrow button until my boot entry is at the top.
  10. The final boot menu would look something like this. Press Apply to save it and return to the main screen.
  11. At the main screen, press OK to save all the changes to the disk.

That's it! After rebooting the machine, there is a new boot entry to boot into Grub2Win besides Windows. And we can boot into Ubuntu with it.

Please note that you may need to update your boot configurations (i.e Update the file names for the kernel and initrd) after a kernel upgrade.

UPDATE: Thanks for the suggestions from the author of Grub2Win, the Grub boot script is now modified to fix some possibile issue on some graphic cards.

14 comments:

  1. Thanks Hopkins for your blog, I successfully installed ubuntu on my 128gb usb drive with 32gb img loopback image.
    The problem is, the "apt-get upgrade" is failed with updating grub related ones, and it seems like the Ubuntu confuses with the boot grub inside the loopback image and real grub at the boot partitions.
    Could you give me some tips or related links on how to deal with this errors?

    ReplyDelete
    Replies
    1. How about just uninstalling the GRUB via apt-get purge? You don't need it anyway.

      Delete
    2. Thank you! I would blacklist the kernel related updates after purging them all.

      Delete
  2. This comment has been removed by a blog administrator.

    ReplyDelete
  3. Thanks for this excellent tutorial for us new devs, however i am running this all but when i run the normal setup after running "try ubuntu" option(after i follow the above process to create /dev/sdx or so), the loop is not visble in the setup. so i cant select and proceed. Any advice?

    ReplyDelete
    Replies
    1. Can you try older Ubuntu releases? is "ls -al /dev/sd*" have your loop device visible?

      Delete
  4. Hello,

    Thanks for this documetation.

    I tried to do the same with Lubuntu 18.04.01 without success.
    The system succeeds in booting, but after a while I have this error:
    Begin: Will now check root file system ... fsck from util-linux 2.31.1
    done
    ALERT! /host/Users/FRNV0210/Documents/data/linux/l1804.img does not exist. Dropping to a shell!

    My grub entry:

    menuentry 'Ubuntu Linux 18.04 ' --class ubuntu --class icon-ubuntu {
    set gfxpayload=1024x768
    set reviewpause=2
    loopback loop0 (hd0,2)/Users/FRNV0210/Documents/data/linux/l1804.img
    set root=(loop0)
    linux /boot/vmlinuz-4.15.0-29-generic root=/dev/sda1 loop=/Users/FRNV0210/Documents/data/linux/l1804.img rw verbose nosplash
    initrd /boot/initrd.img-4.15.0-29-generic
    echo Boot disk address is $root
    echo The boot mode is Partition Address
    sleep -i -v $reviewpause ; echo
    savelast 2 'Ubuntu Linux 18.04'
    echo GNU Grub is now loading Ubuntu Linux 18.04
    }

    Do you have any idea ?

    Thx.

    ReplyDelete
    Replies
    1. I solved my problème by replacing root=/dev/sda1 by root=/dev/sda2 in linux argument. (I have 3 NTFS partitions actually).
      Thanks for this very complete documentation !

      Delete
  5. This comment has been removed by the author.

    ReplyDelete
  6. This comment has been removed by the author.

    ReplyDelete
  7. Hi,

    I have the same problem.

    I managed to boot by doing the following in initrd:
    ______________________________________________________________________________
    mount -t ntfs /dev/'ntfs-partition' /root
    mknod /dev/sda1 b 7 100
    losetup /dev/sda1 /root/'path-to-linux.img'
    exit
    ______________________________________________________________________________
    this will mount the 'ntfs-partition' to /host instead of /root (don't actually understand why)
    I have mounted to root because /host is not present yet
    so now it will see the /dev/sda1 but will not find /host/'path-to-linux-img'
    so run in initrd:
    ______________________________________________________________________________
    losetup -d /dev/sda1
    umount /host
    umount /root
    mount -t ntfs /dev/'ntfs-partition' /host
    losetup /dev/sda1 /host/'path-to-linux.img'
    exit
    ______________________________________________________________________________
    now you should be able to login in the installed image

    Any corrections or explanation why this is happening are appreciated

    ReplyDelete
  8. Doesn't work on ubuntu 20.04 and linux mint 20.

    ReplyDelete