Virtual machine export
This entry reports on step taken to transfer virtual machine (VM) running on FMF server to a new host.
Original setup
On FMF, the virtual machine runs with virsh manager under XEN virtualization.
Disk images
First prepare dd dumps of the disks. In our case, a single disk gets exported, that is /. The exact command was used by Feri, probably something like:
dd if=/dev/virtual-machines/med0/slash of=med0-root.raw bs=512K
The files are then zipped for easier copying:
gzip med0-root.raw
The whole process leaves us with a single file med0-root.raw.gz sized at approximately
7 GB.
Inserting master boot record (MBR)
There are two problems with using raw exported images from XEN. One is, that XEN is much smarter then other virtual machines, but much more tedious to use. The smarter part allows it to pick up the boot images directly from disk without the need for an MBR, acronym for a master boot record. Other virtualization software mimics behavior of non-virtual machines which rely on MBR to help it find operating system images. So we have to create an MBR. An excellent account of all listed procedures can be found Filippo.io[here].
To do that, the disk should have enough space at the begining to put MBR on. Our raw image doesn't have any additional space, so we have to create it by starting a new disk:
dd if=/dev/zero of=new.raw count=1 bs=1MiB
I followed Filippo's advice and checked for disk layout of my laptop:
# sudo fdisk /dev/sda
Command (m for help): p
Units: sectors of 1 * 512 = 512 bytes
[..]
Device Boot Start End Sectors Size Id Type
/dev/sda1 * 2048 206847 204800 100M 7 HPFS/NTFS/exFAT
[..]
Command (m for help): q
#
It seems 2048 sectors of 512 bytes each or 1 MB is enough, which is what I have put in my
dd command. Then we append the rest of the (unzipped) disk:
pv med0-root.raw >> new.raw
pv is a nifty command essentially identical to cat with a progress bar. After 10
minutes, our raw disk has now some unpartioned bytes at its head, and we have to
repartition the image with fdisk:
# sudo fdisk new.raw
[..]
Device does not contain a recognized partition table.
Created a new DOS disklabel with disk identifier 0xd2df1593.
Command (m for help): n
Partition type
p primary (0 primary, 0 extended, 4 free)
e extended (container for logical partitions)
Select (default p): p
Partition number (1-4, default 1): 1
First sector (2048-62916607, default 2048):<Enter>
Last sector, +sectors or +size{K,M,G,T,P} (2048-62916607, default 62916607):<Enter>
Created a new partition 1 of type 'Linux' and of size 47.5 GiB.
Command (m for help): a
Selected partition 1
The bootable flag on partition 1 is enabled now.
Command (m for help): p
Disk new.img: 30 GiB, 32213303296 bytes, 62916608 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0xd2df1593
Device Boot Start End Sectors Size Id Type
new.raw * 2048 62916607 62914560 30G 83 Linux
Command (m for help): w
The partition table has been altered.
Syncing disks.
To add a bootloader, we have to mount our disk to get access to the data. We will use the loop device:
# sudo kpartx -a -v new.raw
add map loop0p1 (253:0): 0 62914560 linear 7:0 2048
# mkdir merlin
# sudo mount /dev/mappar/loop0p1 merlin
# sudo grub-install --target=i386-pc --recheck --debug --boot-directory=merlin/boo /dev/loop0
Installing for i386-pc platform.
[..]
Installation finished. No error reported.
Grub2 does its own configuration, so we will need to chroot into our disk. For that we
will also have to mount some services.
# sudo mount -t proc proc merlin/proc
# sudo mount -t sysfs sys merlin/sys
# sudo mount -o bind /dev merlin/dev
# sudo chroot merlin
Since /etc/resolv/conf might be very different than our host pc, we should make sure the
nameservers are accessible, or we should add our own.
# sudo vi /etc/resolv.conf
# sudo dpkg-query -l | grep grub2-common # if none is find, install
# sudo apt-get install grub2-common
# grub-mkconfig -o boot/grub/grub.cfg
The grub-mkconfig also found bootable sections on my /dev/sda hard drive. We can get rid
of them later.
Some house-keeping tasks - check that disks that are not part of the VM transfer will not
be mounted - do that by commenting out disks which are missing. In our case, /data
partition will not be transfered, so we comment out that line from /etc/fstab. Also,
since boot will depend on it, note the serial number of the disk that will be mounted as
/:
# vi /etc/fstab
# <file system> <mount point> <type> <options> <dump> <pass>
UUID=2f2e0fb3-72e7-454e-ade2-56612df86b99 / xfs defaults 0 1
UUID=8fd49631-5a8d-467f-bbcf-93aa2a471175 none swap sw 0 0
# comment out data
#UUID=c3bd9b07-9073-48f8-9e20-478668c4511d /data xfs defaults 0 1
Also, to not swamp IP address, make sure that network is set to dhcp:
# vi /etc/network/interfaces
[..]
#Primary static address
auto eth0
#set to dhcp
iface eth0 inet dhcp
#Autoconfigured IPv6 interface
iface eth0 inet6 auto
The setup is done. Clean and exit
# exit
# sudo umount merlin/proc merlin/sys merlin/dev merlin
# kpartx -d -v new.raw
del devmap: loop0p1
loop deleted /dev/loop0
# rmdir merlin
# mv new.raw merlin.raw
Changing format of the raw disk
The second problem is that VirtualBox (and other user-friendly virtualization software) can't use raw disks directly. VirtualBox comes with a set of tools that make transition relatively painless. The fact is, however, that we need another space for another full disk.
To change to VDI format, we run:
# VBoxManage convertdd merlin.raw merlin.vdi --format vdi
VDI is a s short name for VirtualBox Disk Image. It is the format of files used to relay virtual disks for VirtualBox desktop virtualization manager. Since VirtualBox is open source, it runs everywhere and will be used as a translator of raw disk images to virtual machine packages that are easy to transport and reuse.
The last step is that VBoxManage will stamp merlin.vdi with a random identification
number, UUID. To match the one set in /etc/fstab we have to correct it by hand:
# VBoxManage internalcommands sethduuid merlin.vdi 2f2e0fb3-72e7-454e-ade2-56612df86b99
Notice that the used number matches the one read from /etc/fstab.
Getting the swap disk
Create an empty space and format it as swap.
# dd if=/dev/zero of=swap.raw count=1 bs=2GiB
# cp swap.raw swap1.raw
# pv swap1.raw >> swap.raw
# pv swap1.raw >> swap.raw
# pv swap1.raw >> swap.raw
# sudo fdisk swap.raw
Welcome to fdisk (util-linux 2.27.1).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.
Command (m for help): n
Partition type
p primary (0 primary, 0 extended, 4 free)
e extended (container for logical partitions)
Select (default p): p
Partition number (1-4, default 1):
First sector (2048-16777183, default 2048):
Last sector, +sectors or +size{K,M,G,T,P} (2048-16777183, default 16777183):
Created a new partition 1 of type 'Linux' and of size 8 GiB.
Command (m for help): t
Selected partition 1
Partition type (type L to list all types): 83
Changed type of partition 'Linux' to 'Linux'.
Command (m for help): p
Disk swap.raw: 8 GiB, 8589918208 bytes, 16777184 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0xe9aa9bd9
Device Boot Start End Sectors Size Id Type
swap.raw1 2048 16777183 16775136 8G 83 Linux
Command (m for help): w
The partition table has been altered.
Syncing disks.
Now we have to mount it to format it:
# sudo kpartx -a -v swap.raw
add map loop0p1 (253:0): 0 16775136 linear 7:0 2048
# sudo mkswap /dev/mapper/loop0p1 -U 8fd49631-5a8d-467f-bbcf-93aa2a471175 -L swap@merlin
Setting up swapspace version 1, size = 8 GiB (8588865536 bytes)
LABEL=swap@merlin, UUID=8fd49631-5a8d-467f-bbcf-93aa2a471175
# sudo kpartx -d -v swap.raw
del devmap : loop0p1
loop deleted : /dev/loop0
Convert it to the vdi format for VirtualBox:
# VBoxManage convertdd swap.raw swap.vdi --format vdi
# VBoxManage internalcommands sethduuid swap.vdi 8fd49631-5a8d-467f-bbcf-93aa2a471175
Create a dummy data disk & partition
Create a small data partition for testing. Create it via
# dd if=/dev/null of=data.raw bs=10M seek=1024 count=0
# fdisk data.raw
Command (m for help): n
Partition type
p primary (0 primary, 0 extended, 4 free)
e extended (container for logical partitions)
Select (default p): p
Partition number (1-4, default 1):
First sector (2048-20971519, default 2048):
Last sector, +sectors or +size{K,M,G,T,P} (2048-20971519, default 20971519):
Created a new partition 1 of type 'Linux' and of size 10 GiB.
Command (m for help): w
The partition table has been altered.
Syncing disks.
# sudo kpartx -a -v data.raw
# sudo mkfs -t xfs -L data /dev/mapper/loop0p1
# sudo xfs_admin -U c3bd9b07-9073-48f8-9e20-478668c4511d /dev/mapper/loop0p1
# sudo kpartx -d -v data.raw
# VBoxManage convertdd data.raw data.vdi --format vdi
The same UUID as in /etc/fstab was created.
Configuring the VM
Now we are ready to run the virtual machine through VBox. When creating the machine,
select "Use existing disk" when prompted for disk configuration and navigate to
merlin.vdi. Then, before you start the machine, go to Storage part of the
configuration and add swap.vdi. Start the machine.
Keep network settings at NAT, under Advanced/Port Forwarding add:
- Name:http, Protocol: TCP, Host Port: 38080, Guest Port: 8080
- Name:ssh, Protocol: TCP, Host Port: 3022, Guest Port: 22
Leave other variables blank.
Adding XPRA for X usage in bare server
Install xpra via apt:
# apt-get install xpra
# apt-get install libpulse-mainloop-glib0 libnss3 libasound2
Since Xorg is picky, go to xpra configuration file and modify link to executable to
point to /usr/lib/xorg/Xorg rather than the default /usr/bin/Xorg which is in path:
# vi /etc/xpra/xpra.conf
[..]
# Selecting virtual X server:
xvfb = /usr/lib/xorg/Xorg -noreset -nolisten tcp +extension GLX +extension RANDR +extension \
RENDER -auth $XAUTHORITY -logfile \
${HOME}/.xpra/Xorg.${DISPLAY}.log -configdir \
${HOME}/.xpra/xorg.conf.d -config /etc/xpra/xorg.conf
[..]
Then one can start an application that requires X by stating:
xpra start :100 --start-child="your-command" --exit-with-children
Install DICOM server
Get Slicer. Unpack and link:
# cd software/install
# tar xvzf [..]/Slicer-X.Y.[..].tar.gz
# ln -s Slicer-X.Y.[..] Slicer
Get labkeySlicerPythonAPI and dicomServer as nixUser:
# cd ~/software/src
## if you have access to f9.ijs.si:
# git clone studen@f9pc22.ijs.si:~/public_html/git/serverDicom.git
## public access
# git clone https://www-f9.ijs.si/~studen/git/serverDicom.git
# git clone git@wiscigt.powertheword.com:andrej.studen/labkeySlicerPythonAPI.git
Run Slicer once as tomcat8 so that .ini files get created:
# export DISPLAY=:100;
# xpra info $DISPLAY > ~/.xpra/temp.log || xpra start $DISPLAY;
# /home/nixUser/software/install/Slicer/Slicer
QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-tomcat8'
Switch to module: "Welcome"
# Ctrl-C
Find the latest Slicer-NNNN.ini file in ~/.config/NA-MIC directory, edit
Additional Paths variable:
# vi ~/.config/NA-MIC/Slicer-27952.ini
[..]
[Modules]
AdditionalPaths=/home/nixUser/software/src/labkeySlicerPythonAPI/labkeySlicerPythonExtension
IgnoreModules=@Invalid()
[..]
Add script startDicom.sh to ~tomcat8/bin:
#!/bin/bash
UHOME=/home/nixUser
DICOMSERVER=$UHOME/software/src/dicomServer
PYTHONPATH=$PYTHONPATH:$DICOMSERVER
SLICER=$UHOME/software/install/Slicer
DICOMDATABASE=/data0/dicom/database/server.sql
DISPLAYID=100
export DISPLAY=:$DISPLAYID
TEMP=~/.xpra/temp.log
#make sure xpra is running a virtual buffer at DISPLAY
xpra info $DISPLAY > $TEMP || xpra start $DISPLAY;
$SLICER/Slicer --no-main-window --python-script $DICOMSERVER/dicomServer.py $DICOMSERVER $DICOMDATABASE
Resizing the data disk
After resizing, sizes above 2TiB can be encountered, where dos/MBR partition table fails. First we switch from MBR to a more recent version of partioning table called GPT.
First we unmount the disk:
>>sudo umount /data
Status:
>> sudo fdisk /dev/sdc
Command (m for help): p
Disk /dev/sdc: 3 TiB, 3298534883328 bytes, 6442450944 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0xaf352020
Device Boot Start End Sectors Size Id Type
/dev/sdc1 2048 4273492457 4273490410 2T 83 Linux
We will have to delete the /dev/sdc partition and replace it with a gpt partition
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.
The size of this disk is 3 TiB (3298534883328 bytes). DOS partition table format can not be used on drives for volumes larger than 2199023255040 bytes for 512-byte sectors. Use GUID partition table format (GPT).
Command (m for help): g
Created a new GPT disklabel (GUID: 81F2CA1E-E1D0-41F3-A42F-0F7360ABCE52).
Command (m for help): p
Disk /dev/sdc: 3 TiB, 3298534883328 bytes, 6442450944 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 81F2CA1E-E1D0-41F3-A42F-0F7360ABCE52
Command (m for help): n
Partition number (1-128, default 1):
First sector (2048-6442450910, default 2048):
Last sector, +sectors or +size{K,M,G,T,P} (2048-6442450910, default 6442450910):
Created a new partition 1 of type 'Linux filesystem' and of size 3 TiB.
Partition #1 contains a xfs signature.
Do you want to remove the signature? [Y]es/[N]o: N
Command (m for help): p
Disk /dev/sdc: 3 TiB, 3298534883328 bytes, 6442450944 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 81F2CA1E-E1D0-41F3-A42F-0F7360ABCE52
Device Start End Sectors Size Type
/dev/sdc1 2048 6442450910 6442448863 3T Linux filesystem
Some complaining:
The partition table has been altered.
Calling ioctl() to re-read partition table.
Re-reading the partition table failed.: Device or resource busy
The kernel still uses the old table. The new table will be used at the next reboot or after you run partprobe(8) or kpartx(8).
Install parted if it is not already there and run
>>sudo partprobe
Command gives no output, but the disk sizes are changed:
Disk /dev/sdc: 3 TiB, 3298534883328 bytes, 6442450944 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 81F2CA1E-E1D0-41F3-A42F-0F7360ABCE52
Device Start End Sectors Size Type
/dev/sdc1 2048 6442450910 6442448863 3T Linux filesystem
Now we can grow the XFS file system and we are done. It seems that /data mounted quitely during one of the previous command, since:
>> df -h
Filesystem Size Used Avail Use% Mounted on
udev 3.9G 0 3.9G 0% /dev
tmpfs 799M 8.7M 791M 2% /run
/dev/sda1 30G 18G 13G 57% /
tmpfs 4.0G 4.0K 4.0G 1% /dev/shm
tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs 4.0G 0 4.0G 0% /sys/fs/cgroup
/dev/sdd1 100G 83G 18G 83% /data1
tmpfs 799M 0 799M 0% /run/user/1001
/dev/sdc1 2.0T 2.0T 75G 97% /data
Now we do:
>>sudo xfs_grow /data
which results in:
>>df -h
/dev/sdc1 3.0T 2.0T 1.1T 64% /data