Customized ubuntu-amd-64 image not installing

[aymanim]

Here are the basic repro steps:

Base image: Ubuntu LTS 12.04
Visualization: VirtualBox
Customizer version: 3.2.3

In virtualbox, I create a new VM and installed the vanilla ISO (from ubuntu.com). It installs as expected.

In Customizer, I "Select ISO" of the same ISO and click "Rebuild ISO" without any modifications. It produces another iso which I use to install on another VM in virtualbox.

During the installation, I get the following error in the copying files phase:

Invalid DOS Header Magic
Can't load image file /target/boot/vmlinuz-3.2.0-51-generic

Partial stack trace:
File "/usr/share/ubiquity/install.py" line 739 in install.run()
File "/usr/share/ubiquity/install.py", line 134, in run self.copy_all()
....

Another discrepancy is that during the original ISO installation, Ubuntu asks for 4.4 GB of free space. During the customized ISO installation, Ubuntu asks for ~1.8 GB of free space.

[ghost]

The descriptions are somewhat vague.

If I understand well:

• you had a 64-bit Ubuntu 12.04 ISO
• you had installed that ISO in new VM in VirtualBox [*1]
• you had installed Customizer in that new VM [*2]
• you had loaded an ISO into that Customizer [*3]
• you didn't modify anything, just directly click on "Rebuild ISO"
• you sucessfully produced ISO from Customizer
• you use that produced ISO to install in another new VM in VirtualBox
• during installation, you got errors as described

And assuming above steps are correct,
we also need to know:
[_1] what host system are you using: 32 or 64-bit?
[_2] which branch did you use: master, gambas, devel?
and how did you install: using script, or via PPA?
[*3] what ISO did you use to load into Customizer: the same vanila ISO?

KIndly answer these questions first.

MUBIIN
[email protected]

[aymanim]

[1] Host: Ubuntu Desktop 12.04 LTS x64

[2] Branch: master. Customizer was installed with the dropbox script found here: https:/clearkimura/Customizer/wiki

Note: This installs Customizer to /opt/Customizer which requires superuser access

[3] I was using the same ISO directly from the Ubuntu website: ubuntu-12.04.2-desktop-amd64.iso.

Your understanding is accurate. Some more info: I have tried working directory as /home (default) and also /home/USERNAME/ to isolate permissions issues.

As I mentioned before, a related issue is probably causing the size discrepancy as well. This is a screen shot of the requirements page for the ISO created with Customizer

It asks for at least 1.2 GB available drive space. A vanilla installation for Ubuntu 12.04 Desktop requires ~4.4 GB available drive space

In the "Copying Files" step during installation, this is the error screen:

In the console output, the above mentioned error is described.

[ghost]

So you are running a 64-bit host and using 64-bit ISO.
And you are using stable master branch.
And you sucessfully able to load the LiveCD before installation.

My pre-assumption is, Customizer doesn't seemed to be the problem here.

Since you reports installer crash and disk size discrepancy issue, I
estimate this issue has something to do with the Ubiquity installer or
the VirtualBox emulator.

Here, I will have to ask other questions as well:

[4] Are you using the latest VirtualBox release for your 64-bit host
i.e. via PPA or DEB downloaded from the VirtualBox official site?

[5] Do you have enough virtual disk space created in VirtualBox i.e. 5 GB VDI?

[6] Do you have enough virtual memory set in VirtualBox i.e. 512 MB RAM?

If your answer is all "YES" for above additional questions,
try either one of solutions below (one by one):

[a] disconnect from Internet before installation

[b] choose "Try Ubuntu without installing" menu when booting LiveCD

[c] before installation, remove "ubiquity-slideshow-ubuntu" package
from Software Center or Synaptic or simply by typing in Terminal:
sudo apt-get remove ubiquity-slideshow-ubuntu
Then only run the installer.

See if either one can solve your issue?

MUBIIN
[email protected]

[fluxer]

Did you choose to delete the boot files?

[fluxer]

Is the issue solved and if yes did you found the cause and solution for it?

[ghost]

I'm closing this issue, since there is no reply from user for quite some time. Unless there is a reply, anyone can choose to reopen this issue if this issue persists.

Marked as Invalid.

[fluxer]

The second issue @aymanim described (see the last two sentences from his initial report) may be true, maybe Ubuntu changed the way the filesystem size is stored (different units?) but I'm not going to test that.

[ghost]

Recent Ubuntu releases calculates the disk size using base 10. Ubuntu 10.10 or newer will now see 1 MB as 1000 bytes. Ubuntu 10.04 or older will still see 1 MB as 1024 bytes.

Aside from above, there are few other unknowns:

• which version of VirtualBox emulator did @aymanim use?
• does 64-bit ISO rebuilded on 64-bit host only works when emulated
  on 64-bit VirtualBox emulator?
• which hard disk format did @aymanim use: VDI, VDMK, VHD, HDD?
• does VirtualBox filesystem contributed to disk size calculation error?

.. and the list goes on.

I had never used 64-bit machine before, so I can't reproduce this issue anyway.

What I can do is, to check the recent Ubuntu release ( i.e. 12.04 ) 32-bit ISO rebuilded on 32-bit host. So far I have only checked the rebuilded ISO can be boot as Live CD on VirtualBox, but I haven't try to install on VirtualBox disk yet.

[ghost]

I don't have Ubuntu ISO images from 10.10 to recent releases because the free LiveCD service has stopped. And I don't download Ubuntu ISO image file anymore because it is now larger than CD size, not suitable for slow connection user like me.

Tested Xubuntu 12.04 32-bit ISO rebuilded in Customizer 3.2.3 instead.
Reproduced steps as follows:

• Succeed to run as Live CD in VirtualBox 32-bit host.
• Ubiquity installer shows 1.4 GB free disk space required.
• Do fresh install to a fixed volume 5 GB VirtualBox ( VDI ) disk image.
• Ubiquity installer did not crash
• Able to reboot and enter into installed system

What can I tell here:

• Other Ubuntu derivatives do not encounter this issue
• Ubiquity installer for Ubuntu derivatives does not crash

And as matter of fact:

• It is true that Ubuntu 12.04 Desktop requires about 5 GB free disk space
• It is true that Ubiquity installer for 32-bit has issue
• But no issue was found with 64-bit installer, according to this comment at askubuntu thread below:

http://askubuntu.com/questions/125280/what-are-the-minimum-system-requirements-for-12-04-lts/125282#125282

Verdict:
I cannot reproduce this issue due to lack of stuffs.
This issue will remain invalid until other users encounter and able to reproduce this issue.

[ghost]

This issue is found to be "Invalid" in earlier findings.
Added "Pending" label, because this issue still lack of information.
I have reopened for other users' feedback.

[ghost]

Some accidential findings indirectly related to this issue, while looking into just solved issue #37 by @fluxer last weekend.

Latest Customizer (gambas3) has been installed successfully on Xubuntu 12.04 (32-bit) virtual machine, under VirtualBox 4.3 (32-bit) emulator, with virtual hard disk created using VDI file format.

Case 1: Fixed allocated VDI

• will be recognized by virtual system as "normal" disk
• boot and login normally
• Customizer able to extract selected ISO properly

Case 2: Dynamically allocated VDI (cloned and resized from Case 1)

• no difference, in terms of system recognition
• boot is abnormal--shows fixing errors at every reboot time but still able to login
• Customizer able to extract, but failed to proceed at random point of progress
  and produced "[x] Unable to unsquash.." like error message

Early findings show that virtual hard disk type has effect on application (Customizer) behaviour when extracting ISO to working directories. However, this needs to be compared with Xubuntu 12.04 (32-bit) on real machine, for above justification. To be confirmed later.

[ghost]

Follow up on accidential findings, while double-confirming issue #37 just now.

Xubuntu 12.04 (32-bit) has been reinstalled on a newly created, "fixed allocated" VDI hard disk.

• Installation of host system is normal.
• Installation of Customizer is normal.
• Extracting ISO is also normal; Successfully extracted Ubuntu Mini Remix 10.04 32-bit, Ubuntu Mini Remix 12.04 32-bit, Linux Mint 13 Cinnamon 32-bit, in turns.

Even without testing on real machine, this reproduction of Case 1 strongly suggests that user should always use "fixed allocated" VDI hard disk whenever using Customizer in virtual machine.

[fluxer]

Maybe this is a bug with ubiquity, you should try with newer version. Try to fetch the ubiquity packages for other release and install them manually in the chroot environment. If that helps then it's a bug in ubiquity.

[ghost]

Recently, I experienced this issue while continue to test drive Customizer from devel branch. I had tried out with the following setup.

• minimal XFCE remix based on UMR 12.04 LTS 32-bit ISO image file
• using VirtualBox OSE on 12.04 LTS 32-bit host system
• using an 8GB "fixed allocated" VDI hard disk

Even I had already installed ubiquity packages without recommended packages (means without the suspected problematic ubiquity-slideshow-ubuntu package), this issue still occurred.

One similar symptom is, the discrepancy of free disk space.

• in @aymanim case, 1.8 GB free disk space is required using 64-bit ISO on 64-bit host system
• in my case, 700+MB free disk space is required using 32-bit ISO on 32-bit host system
• normal installation will ask 4.4 GB free disk space

At this point, I am unsure if the required disk space shown in Ubiquity installer is actually depends on the ISO image file size in use? This makes sense, because the remix I had rebuilded is about 300+ MB and required free disk space is 700+ MB, which is about 2.5 times of ISO image file size.

Besides this, my installation will fail after the installer had shown "almost finished installation" message. I have confirmed that disk has been formatted and typical Linux FileSystem structures are present on the disk. Despite this, I was unable to boot into Live session.

Quick googling also reveals that few users filed similar bug under ubiquity package on Ubuntu. But, most of them remain open or closed due to inactivity, without any clear resolution.

I won't jump to conclusion yet, as I am still trying to find any workaround.

Many things I am yet to try: using latest version of its dependency packages especially python-apt package; look into its python script files, based on error log dumped into /var/log/syslog.

I will follow up this issue for time being, but on irregular basis due to my working condition.

[fluxer]

I will try to reproduce in the near future but I can not promise anything. If you can share the ISO you had problems with that would be great, I know that your internet connection is not very good so worry not if you can't.

[fluxer]

I found the cause for the issue about the disk space requirement. Fix should be submitted shorty, I'm testing right now to ensure the fix works as it should.

[ghost]

@fluxer
That was unexpected! I was confident that the free disk space issue caused by ubiquity package.

Moreover, users whom had filed bugs at launchpad are those supposedly using official ISO images. In contrast,
I am using own remix build ISO image.

After all, @aymanim experienced this issue using master/ gambas3 branch. But I failed to reproduce this issue using these stable branches (as reported in earlier comments).

I'm curious. What leads you find this issue is related to Customizer now?

Sent using a Sony Ericsson mobile phone

[fluxer]

@clearkimura
Since Python is programming language for Humans (IMHO) it should be straightforward that in the commit that is linked after my previous comment I adjust the rebuild behavior of Customizer to store the size of the root filesystem (e.g. /home/FileSystem) instead of the size of the SquashFS filesystem. The way I see it Ubiquity doubles the number stored in casper/filesystem.size that you can find (or should) on any Ubuntu based ISO image and sets that as a minimum required disk space. Even now the method used does not reproduce the free disk space requirements and I have no idea why. From my tests with the offical (unmodified) Xubuntu ISO the number was 4.5GB, and on the one customized it was 3.5GB IIRC (I was too sleepy, not enought sleep lately). Both the official and modified ISO use bytes to store the size but there is one thing that I think may be the cause - Customizer does not "walk" trough symlinks (see https:/clearkimura/Customizer/blob/d7136472a7ed24b6b9307e395195a01208171d5a/src/actions/rebuild.py#L154). And to get you the results from my experiment:

Official 12.04 amd64 Xubuntu ISO filesystem.size content - 2257629184
Customized 12.04 amd64 Xubuntu ISO filesystem.size content - 2049673133

I will test further in a few days once I have more free time and see if I can further improve on that commit.

And to let you know about the Gambas2/3 branches - they are not supposed to work with later release, as I've said before - things change. I have not used those branches for quite a while.

Above all, my so called fix is supposed to fix a side issue - the one that makes Ubiquity require less free disk space than it should. It may or may not fix the issues with the actual installation, I really doubt it will but is worth the shot since one error can cause many more.

[ghost]

Some follow up from my previous comments:

Besides this, my installation will fail after the installer had shown "almost finished installation" message.

I found the cause of this installation failure by ubiquity package. It was due to the suspected python-apt package dependency is not using the correct version of package.

The installation will fail when using ubiquity version 2.10.29 (updated version) against python-apt version 0.8.3ubuntu7 (default version). The fix is simply upgrade the python-apt package to version 0.8.3ubuntu7.2 (updated version).

Apparently, this issue is already reported by someone on launchpad, but nobody really attend it when I found the thread last time.

That was unexpected! I was confident that the free disk space issue caused by ubiquity package.

As for this free disk issue, I got mixed up with the installation failure issue. My bad.

Regardless of the free disk requirement, I was able to do installation to an 8GB "fixed allocated" VDI hard disk (same environment setup as explained in earlier comment).

Except, I am still using the older commit 98cc689 to do the successful installation.

[fluxer]

So, we can consider this a issue that is not our own per-se.

@clearkimura
I would like to look into the free disk space requirement one more time and I guess after that we can close this issue, agreed?

[ghost]

So, we can consider this a issue that is not our own per-se.

Apparently. With one exception: the original issue submission by @aymanim case. We couldn't figure out what was all that about. It may have been resolved in other workaround, hopefully.

@fluxer
Agreed. I think you are well aware, but just my one-yen advice: don't take the issue of free disk space requirement seriously. We don't need to fix something that isn't broken.

[fluxer]

With one exception: the original issue submission by @aymanim case. We couldn't figure out what was all that about. It may have been resolved in other workaround, hopefully.

I suspect that the original issue mentioned in the first post by @aymanim is due to boot files being deleted, which is no longer supported by the devel version (for a good reason) which causes more troubles than any good since most users used it without reading the documentation you provide where (if I remember correctly) there is a fat warning about it that it will break the installation process.

[ghost]

This issue shall be closed, because "old stable" will be deprecated.

For future issues, please submit to official fork: https:/kamilion/customizer

P.S.: The official fork was removed, then "old stable" repo will be transferred to fork maintainer.