In a previous article, I answered the question: If VMware ESXi 4 is so small, why is it so big? It’s quite clear now that the disk footprint of VMware ESXi 4 is less than 60MB. But to really drive the point home, I wanted to demonstrate that VMware ESXi 4 could boot and run from a tiny 64MB flash device, so I asked Olivier Cremel, the inventor of ESXi, if that was feasible. He said it was — and gave me advice on how to set it up. This article shows you how.
Requirements
- The VMware ESXi 4 dd image file (Vladan’s site may help)
- A 64MB flash drive — check your junk drawer
- A Linux machine — make sure you have mkfs.vfat
- This fdisk script — save it as a txt file
Extract necessary components from the large image
Use dd to copy the first 1.5MB of the boot partition and the files that make up ESXi.
# dd if=VMware-VMvisor-big-175625-x86_64.dd of=esxi-boot-part skip=32 count=3000
3000+0 records in
3000+0 records out
1536000 bytes (1.5 MB) copied, 0.036249 seconds, 42.4 MB/s
# mkdir tmp
# mkdir esxi
# mount -o loop,offset=$((8224*512)) VMware-VMvisor-big-175625-x86_64.dd tmp/
# cp tmp/* esxi
# sync
# umount tmp/
Prepare the flash device
Plug in your flash device and use dmesg to determine the device id (/dev/sdk in this example). Run fdisk on it and delete all partitions – very important. Then, run my fdisk script to create the required partition structure. If this doesn’t work on your flash drive due to different size/geometry, you will need to create the partitions yourself — refer to the table at the end of these instructions for reference.
# fdisk /dev/sdk < fdisk.txt
Write data to the flash device
Use dd to write the saved boot partition to flash.
# dd if=esxi-boot-part of=/dev/sdk4
3000+0 records in
3000+0 records out
1536000 bytes (1.5 MB) copied, 1.31462 seconds, 1.2 MB/s
# sync
Then create a FAT filesystem on the flash drive, mount it, and copy the ESXi files.
# mkfs.vfat /dev/sdk5
mkfs.vfat 2.11 (12 Mar 2005)
# mount /dev/sdk5 tmp/
# cp esxi/* tmp/
# sync
# umount tmp/
Double-check your partition layout
If your disk partition does not look like this, something went wrong.
# fdisk -l /dev/sdk
Disk /dev/sdk: 65 MB, 65273856 bytes
3 heads, 42 sectors/track, 1011 cylinders
Units = cylinders of 126 * 512 = 64512 bytes
Device Boot Start End Blocks Id System
/dev/sdk1 26 1011 62118 5 Extended
/dev/sdk4 * 1 25 1554 4 FAT16 <32M
/dev/sdk5 26 1011 62097 6 FAT16
Test it out
If you followed all of the instructions above, you should be able to boot from that flash drive.
There you have it -- a fully functional VMware ESXi 4 system entirely contained on a 64MB flash device. Of course, this procedure is entirely unsupported, so please do not use a system like this in production. Please!
Oh!! This is great info, will have to try this at work!
You means ESXi is french ?!
This great. It really does definitively burys any M$ FUD surround this argument. But I wonder if we and Microsoft have been looking at ESXi for the completely wrong angle. On the plane over to San Francisco this week – myself and Tom Howarth began to discuss ESXi short-comings. Anyway, I’m think when I get back to the UK next week – that I will sit and dissect the problems with ESXi. Interested in bouncing some ideas about?
I’d be interested, of course. Probably see you at VMworld.
Pingback: Twitted by Jimmy_kehoe
i try it before and failed. now my flash disk only shown to have 4MB. is there anyway to wipe the flash disk? originaly it’s 2GB.
thanks.
you can use HP Drive Key Boot Utility to wipe it
Pingback: Le concepteur d’ESXi est Français ! - Hypervizor.fr
I realize this is a late comment but:
Can you elaborate on why you performed some of these steps? For example- why do you skip the first 32 blocks? Why 1.5MB? is that the size of the files in the partition? The first partition itself is actually 4 MB in size so why 1.5MB?
The partition table entries are not in disk order. Run fdisk -u -l VMware-VMvisor-big-175625-x86_64.dd to see what I mean.
On my “64MB” USB stick I did not have access to a full 64MB — typical for storage devices of all kinds, so I had to trim that partition down a bit. 1.5MB was chosen arbitrarily and it worked.
I’ve been able to get esx to boot on a key using this method- but no changes I make to the system are being saved to the key. If I reboot- the changes are lost. Are you able to save changes to the key across reboots?
This is an exercise to show that the core functionality of ESXi is fully contained in 64MB and not 1GB as some have claimed.
I understand it’s meant as a demo but a demo that doesn’t really work isn’t nearly as impressive as one that doe. In my case I have lots of 256 MB USB keys I’d like to give to people to get them to try esxi- but it’s a lot less useful if the config doesn’t get saved.
I’m not really sure why it wouldn’t save the config- everything is definitely there- esxi boots- vm’s run- etc. It just seems like the memory filesystem isn’t being written out to disk.
I actually had a 256MB key so I went a little further with my test: I set up a partition for VMWare tools and installed those to the key (I left out Vsphere client as I can download and install the latest version from vmware’s site easily enough and at 111MB it would have taken up too much room). The tools worked fine and let me install to the vm’s I tested.
I’d still like to figure out the save issue if anyone has any ideas.
Nice work Eric. Great post. Used this method (a la a bit modified to extract the VMware-VMvisor.dd.bz2 on a Linux box) and I also used a 128 MB flash stick I had laying around. I must admit this is very cool to prove the core thin hypervisor is indeed a very small footprint and I appreciate your explanations in your two-part articles explaining the use of the other partitions. Doesn’t seem to load as quick (but this may be because it is a 1.x USB device and not 2.x complaint like most of my 1-2 GB ESXi usb sticks I test with) but all functionality is there minus of course tools/vSphere client (web interface give a cute blank page as I assume it is missing tomcat portlet)/etc. Booted up a xppro VM that I recently installed the openssh for windows package on for test I was doing for a friend (I know it seems so bastardized) and yanked the 128MB ESXi u1 stick and to my immediate surprise my ssh session into that xppro box stayed running as did the VM. Must have been because the ESXi hypervisor is loaded into memory. Powered off my laptop running ESXi that joins my two way cluster and approx 45 seconds later HA fired the VM back up on esx2 node. Very cool indeed! Also noted the settings do not stick but that is no surprise with this stripped/slim ESXi install.
Keep up the good work!
Can this work on a normal laptop for testing purposes.But with no CPU AMD-V
can it install on a usb flash, then use dd to backup the sda4 and sda5,
and then use this 2 patition backup file to restore to a 64MB flash with your fdisk script?
Patrick, you could use a 1GB USB as an intermediate source as you suggest. That’s initially how I approached this, but figured it would be easier to not require two different USB sticks.
Eric
the sd*4 seems large the 1.5M and the data inside it seems less than 512K?
dd if=VMware-VMvisor-big-175625-x86_64.dd of=esxi-boot-part skip=32 count=3000 should not contain all the sd*4? It is work?
if the data is less than 500K, can I use dd if=VMware-VMvisor-big-175625-x86_64.dd of=esxi-boot-part skip=32 count=1000 ?
has you test the VMware-VMvisor-Installer-4.0.0.Update01-208167.x86_64.iso, is it can store in a 64M disk?
can it use the same way to copy the file to sd*4?
mount -o loop,offset=$((32*512)) VMware-VMvisor-big-175625-x86_64.dd tmp/
Patrick,
Please keep in mind that this is just a proof-of-concept and not intended for real-world use. The point was to demonstrate that ESXi is very small despite the “huge” 1GB flash requirement. Given the difficulty in even finding a sub-1GB USB flash device these days, there is not much value in getting it to run on 64MB.
When I put together the prototype, I stopped when I hit on a repeatable formula that works. I’m sure there are ways to optimize, feel free to tinker around and post your findings if you come up with a cool solution.
Thanks for reading VCritical.
Eric
Hello
Here is another way to do it:
I use the fantastic boot loader grub4dos and boot ESXi with this entry:
title Vmware ESXi4
find –set-root /grldr
kernel /ESX4/vmkboot.gz
module /ESX4/vmk.gz
module /ESX4/sys.vgz
module /ESX4/cim.vgz
module /ESX4/oem.tgz
module /ESX4/license.tgz
module /ESX4/mod.tgz
It boot from a pendrive with NTFS filesystem.
You only need to install grub4dos on the pendrive
and take these files from the ESXi dd image.
This way you can boot ESXi, Windows, Linux, etc. from ONE pendrive.
Greetings!