WubiJaunty

Differences between revisions 11 and 38 (spanning 27 versions)
Revision 11 as of 2008-05-20 12:15:21
Size: 6360
Editor: netmail
Comment:
Revision 38 as of 2008-12-11 20:25:00
Size: 7224
Editor: 216
Comment:
Deletions are marked like this. Additions are marked like this.
Line 2: Line 2:

 * '''Launchpad Entry''': UbuntuSpec:wubi-python
 * '''Created''': 2008-12-06
 * '''Contributors''': AgostinoRusso
Line 5: Line 9:
This page is for discussion related to Wubi development in the Intrepid cycle. This page is for discussion related to Wubi development in the Jaunty cycle.
Line 7: Line 11:
== Release Note == == Wubi Frontend ==
Line 9: Line 13:
Further improve the user experience when using Wubi. === Wubi frontend rewrite ===
    
Current nsis code is difficult to maintain and extend. But there is a trade-off in terms of size increase. Moving to python will simplify maintenance, it will make it easier to incorporate new features (in primis bittorrent support) and it will open the door to wubi ports to other platforms.
Line 11: Line 17:
== Kernel Related Changes == A preliminary implementation (although not complete yet) is available in https://code.edge.launchpad.net/~ubuntu-installer/wubi/jaunty.python
Line 13: Line 19:
 * ntfs-3g fsck (VERY IMPORTANT!!!)
 * Loopinstallation speed: (http://lkml.org/lkml/2008/1/9/50)
 * Loopinstallation robustness (https://bugs.launchpad.net/wubi/+bug/204133),
 * Remounting of loopfiles (https://bugs.launchpad.net/wubi/+bug/201750)
 * Remounting of ntfs3g (https://bugs.launchpad.net/wubi/+bug/186117)
 * Online resizing (in particular squeezing of ntfs partition while it is mounted and even hosting /)
 * Hibernation with swap files (double check suspend with fuse filesystem)
 * fallocate in ntfs/vfat
 * native ntfs installations (last time I checked szaka mentioned it was 2 years away)
To reduce size, the code uses a thin wrapper around ctypes (winui) and a custom python exe packager (pylauncher) that uses lzma solid compression. The code should also make it easier to port Wubi to other platforms in the future.
Line 23: Line 21:
== Possible New Features == * Check accessibility of winui interface with Luke
* Fix tabs and transparency in winui
Line 25: Line 24:
 * d-i support (i.e. same hooks as lupin-casper)
 * Add bittorrent support to built-in download manager
 * Built-in cdburner (need to evaluate CD burning libraries)
 * Bootable USB device with ISO with casper-rw
 * HD-media like installation from HD (Target empty partitions?).
 * Grub2 (as recommended by bean123)
 * Better build scripts (nsis is available in the repositories)
 * Support for "large" distros such as ubuntu studio or edubuntu (support separate add-on CD directly? or leave it as pos-installation task? shall we automate such tasks somehow?)
 * support DVD
 * Use resizable image files (LVM or simply 4GB zeroed + nGB sparse, or qcow2 type of images or images that are contiguous for the first 4GB and sparse for nGB)
 * Support for "native" installations (installation inside a folder, no loopfile)
 * "My Documents" bookmark pointing to the appropriate windows folder (migration-assistant?)
 * Add "Use existing ISO" button, to use a pre-downloaded ISO (at the moment ISO and wubi.exe can be placed in the same folder, but it is not too obvious).
=== Windows GUI chnages ===
Line 39: Line 26:
== UI Changes ==  * Add "Use existing ISO" button during download, to use a pre-downloaded ISO (at the moment ISO and wubi.exe can be placed in the same folder, but it is not too obvious).
 * Keep the accessibility page in sync with the gfxboot one
Line 41: Line 29:
 * Can we further simplify the interface?
 * Changes to the accessibility page
 * Do we want to have advanced options?
 * Possiblility of a frontend rewrite in python or c++ and such
  === Downloader ===

 * Add bittorrent support to the download manager:
  * Try first to use bittorrent
  * If that fails, use http with automatic country mirror selection
 * Add automatic proxy support

== Wubi Backend ==

=== ntfs-3g fsck ===

At the moment unclean shutdowns result in ntfs being flagged as dirty which in turns prevents us from mounting it. The user is then forced to clean the filesystem from within Windows, by booting and rebooting cleanly and sometimes by running chkdsk /r.
It is important to be able to do consistency checks, fix most common problems and if it is safe to do so, reset the dirty flag and mount normally from within Linux.
    
See https://bugs.launchpad.net/ubuntu/+source/initramfs-tools/+bug/226622

=== Grub 2 ===

Evaluate the possibility of using grub-2. In particular that seems to support loop mounting and ntfs, which could greatly simplify the backend design.

=== Loopinstallation speed ===

It should be possible to improve on the stock loop module. See:
 
 * http://lkml.org/lkml/2008/1/9/50
 * http://git.kernel.dk/?p=linux-2.6-block.git;a=shortlog;h=loop-extent_map
 * http://lkml.org/lkml/2008/1/9/335

Colin King seems to like the dm-loop approach, although he mentioned that dm-loop will give better performance only if it has direct access to the block device - in the Wubi case it won't, so the difference is debatable. More testing is required to confirm this.

Also assess whether the new syncio option in ntfs-3g has a negative impact on the use of a swap file.

=== Hibernation with swap files ===

It seems that this feature is already planned for Intrepid because other projects also require it (confirm). At the moment suspend-to-disk works when swap is on file, but resuming does not.

=== Remounting of loopfiles ===

See https://bugs.launchpad.net/wubi/+bug/201750)
 
=== Remounting ntfs-3g ===

See https://bugs.launchpad.net/wubi/+bug/186117

=== D-I support (i.e. same hooks as lupin-casper) ===
 
  * initrd-preseed needs to be able to take UUID and path; possibly file-preseed as well but we really need the former
  * iso-scan needs to be able to take UUID and path
  * sync this with lupin-casper
  * lupin-casper should drop initrd-override
  * casper-premount.sh should be native to lupin-casper

=== HD-media like installation from HD (target empty partitions?) ===

  * Depends on fixing bugs in partman that cause partition tables to be marked dirty
  * Any partition table changes require the disk to be entirely unused during partitioning, which is infeasible in Ubiquity
   * Consider supporting this only for d-i

=== Resizable images ===
 
Add support for resizable virtual disks. This can be done via:
 
 * LVM
 * Use 4GB contiguous (zeroed) + nGB sparse
 * qcow2 type of images

=== MyDocuments and host bookamarks ===

Add bookmarks for MyDocuments and Host drive

  * Evan does not want to add this to m-a. He's afraid users would get horribly confused when they removed their NTFS partition and all of their documents suddenly disappeared. Since it's specific to Wubi, Evan thinks it should be done there.
  * It was agreed that bookmarks are a preferred solution over symlinks or bindmounts.
  * One complication of having a MyDocuments bookmark is that it does not necessarily lie in the /host volume, so that the partition might not be mounted and/or mounted with r/o permissions.

  * see xdg-user-dir

=== Sysvinit improvements ===

 * In umountfs unmount items in /proc/mounts in reverse order stopping at /.
 * Don suggested using lazy unmount (-l) but it should not be required given the above, also lazy unmounts do not help with /host since that contains / which is never unmounted.
 * Keep the -f option, but either fix its behaviour within umount when handling bindmounts (now umount -f bindmount will unmount the underlying device) or keep using WEAKMOUNTPOINTS
Line 48: Line 114:
 * Could use a native installation approach (installation inside a folder, no loopfile)  Support installing inside a folder (no loopfile). Handy for Lubi and Mac ports.
 * very important that this be separated such that it imposes zero cost on most Ubuntu users
 
Line 54: Line 122:
 * Requires D-I hooks.
 * Requires porting upstream all the changes to initramfs-tools, grub & co, sysvinit scripts, caspre/d-i initrd hooks, lupin-support.
 * Investigate the opportuinty of a tighter integration with the win32-loader project (at the moment it does netinstallation only).
 * Requires porting upstream all the changes to initramfs-tools, grub & co, sysvinit scripts, casper/d-i initrd hooks, lupin-support.
  * ... and almost certainly most of Ubuntu's d-i customisations! This is a large project (which should be done anyway, but will take a while and contains some elements which are likely to be contentious).
  * console-setup needs to become the default in Debian
  * localechooser encoding support needs to be fixed
 * Investigate the opportunity of a tighter integration with the win32-loader project (at the moment it does netinstallation only).
Line 58: Line 128:
== Migration Tools (loopinstallation -> real partition) == == Wishlist ==
Line 60: Line 130:
 To be discussed in separate session:
 * Support for loopinstallations within ubiquity (it should detect existing loopinstallations and offer to migrate them)
 * Simplified migration GUI.

=== Migration Page Mockup ===

{{{
Action:
 * Migrate the existing Wubi installation to a real Partition
 * Install Ubuntu
 
Install into:
 * Free space in hard disk #1 (sda, 8GB)
 * Free space in hard disk #2 (sdb, 6GB)
 * Empty partition #2 of hard disk #1 (sda2, 26GB)
 * Empty partition #5 of hard disk #1 (sdab, 12GB)
 * Advanced Mode

Free up space for the new installation:
 [Resize an existing partition] [Delete an existing disk/partition]
}}}

The install-into list only contains safe items, I.E. items that do not contain any file and are large enough.
For instance, if disk #1 is in use, and the user wants to dedicate the full disk #1 to the installation, he will have to click on "Delete an existing disk/partition", delete disk #1 and then select "Free space in hard disk #1" which will appear in the install-into list. This is safer and clearer than merging the two operations into a single action such as "Install into disk #1", as in current ubiquity dialog. In particular, with the above suggestion, it is not possible to accidentally overwrite/delete any file unless the user explicitly selects "Delete an existing disk/partition" or (s)he uses the "Advanced Mode".

{{{
Resize partition dialog:
 Squeeze: [disk #1][partition #1]
 to 5GB [ =====|========= ] and free up 10GB
 -----------------------------------------------------
 Partition Info: /dev/sda1
 Total size: 15GB
 In use: 4GB
 Label: mytarget
 Filesystem: NTFS
 Files: WINDOWS, temp, Program Files, RECYCLER
 -----------------------------------------------------
 [Resize Partition] [Cancel]
}}}

{{{
Delete partition dialog:
 Delete: [disk #1][partition #1 | all]
 -----------------------------------------------------
 Partition Info: /dev/sda1
 Total size: 15GB
 In use: 4GB
 Label: mytarget
 Filesystem: NTFS
 Files: WINDOWS, temp, Program Files, RECYCLER
 -----------------------------------------------------
 [Delete Partition] [Cancel]
}}}

Advanced Mode (same as ubiquity):
 * Overwrite an existing disk/partition #skip disk/partitions where wubi is installed
 * Create multiple mountpoints
 * Support for \"large\" distros such as Ubuntu Studio or Ubuntu Education Edition (support separate add-on CD directly? or leave it as post-installation task? shall we automate such tasks somehow?)
 * support DVD
 * Bootable USB device with ISO with casper-rw
 * Grub2 (as recommended by bean123)
  * grub is still unsupported in Ubuntu anyway, and it won't be critical to have grub4dos unsupported too
  * we should move over eventually, but consensus is \"not yet\"
 * Built-in CD burner (need to evaluate CD burning libraries)
 * Online resizing (in particular squeezing of ntfs partition while it is mounted and even hosting /)
 * native ntfs installations (last time I checked szaka mentioned it was 2 years away)
 * fallocate in ntfs/vfat (we can generate the virtual disk images via the windows frontend)
Line 120: Line 143:
[http://brainstorm.ubuntu.com/search?keywords=Wubi&ordering=mostvotes Wubi in brainstorm]

== Outstanding Issues for 8.04.1 ==

Most have already fixes committed, but need to be tested, approved and released.

 * Bug #217348 Incorrect disk order detection generates wrong menu.lst
 * Bug #222690 install failed during "import document and setting" at 88% - French windows XP
 * Bug #224311 grub4dos gets jammed with the message "Turning On Gate A20"
 * Bug #204128 After install completed bar wasn't all green and installer hung
 * Bug #136682 File with non-C/special characters in host computer disappear
 * Bug #226622 Wubi has unclear error message on NTFS dirty flag
 * Bug #224697 Disable hibernation if swap is on file
 * Bug #223250 Country detection is wrong when country name is localized in the registry
 * Bug #217593 If the installer fails or is aborted, users can end up in Live CD Desktop, which is confusing
 * Bug #207137 Wubi fails to install from my cd-rw but does from my dvd-rw
 * Bug #230703 Casper only scans vfat filesystems for cow files
 * Bug #230716 Allow casper to use a squashfs filesystem within an arbitrary path
[[http://brainstorm.ubuntu.com/search?keywords=Wubi&ordering=mostvotes|Wubi in brainstorm]]

Summary

This page is for discussion related to Wubi development in the Jaunty cycle.

Wubi Frontend

Wubi frontend rewrite

Current nsis code is difficult to maintain and extend. But there is a trade-off in terms of size increase. Moving to python will simplify maintenance, it will make it easier to incorporate new features (in primis bittorrent support) and it will open the door to wubi ports to other platforms.

A preliminary implementation (although not complete yet) is available in https://code.edge.launchpad.net/~ubuntu-installer/wubi/jaunty.python

To reduce size, the code uses a thin wrapper around ctypes (winui) and a custom python exe packager (pylauncher) that uses lzma solid compression. The code should also make it easier to port Wubi to other platforms in the future.

* Check accessibility of winui interface with Luke * Fix tabs and transparency in winui

Windows GUI chnages

  • Add "Use existing ISO" button during download, to use a pre-downloaded ISO (at the moment ISO and wubi.exe can be placed in the same folder, but it is not too obvious).
  • Keep the accessibility page in sync with the gfxboot one

Downloader

  • Add bittorrent support to the download manager:
    • Try first to use bittorrent
    • If that fails, use http with automatic country mirror selection
  • Add automatic proxy support

Wubi Backend

ntfs-3g fsck

At the moment unclean shutdowns result in ntfs being flagged as dirty which in turns prevents us from mounting it. The user is then forced to clean the filesystem from within Windows, by booting and rebooting cleanly and sometimes by running chkdsk /r. It is important to be able to do consistency checks, fix most common problems and if it is safe to do so, reset the dirty flag and mount normally from within Linux.

See https://bugs.launchpad.net/ubuntu/+source/initramfs-tools/+bug/226622

Grub 2

Evaluate the possibility of using grub-2. In particular that seems to support loop mounting and ntfs, which could greatly simplify the backend design.

Loopinstallation speed

It should be possible to improve on the stock loop module. See:

Colin King seems to like the dm-loop approach, although he mentioned that dm-loop will give better performance only if it has direct access to the block device - in the Wubi case it won't, so the difference is debatable. More testing is required to confirm this.

Also assess whether the new syncio option in ntfs-3g has a negative impact on the use of a swap file.

Hibernation with swap files

It seems that this feature is already planned for Intrepid because other projects also require it (confirm). At the moment suspend-to-disk works when swap is on file, but resuming does not.

Remounting of loopfiles

See https://bugs.launchpad.net/wubi/+bug/201750)

Remounting ntfs-3g

See https://bugs.launchpad.net/wubi/+bug/186117

D-I support (i.e. same hooks as lupin-casper)

  • initrd-preseed needs to be able to take UUID and path; possibly file-preseed as well but we really need the former
  • iso-scan needs to be able to take UUID and path
  • sync this with lupin-casper
  • lupin-casper should drop initrd-override
  • casper-premount.sh should be native to lupin-casper

HD-media like installation from HD (target empty partitions?)

  • Depends on fixing bugs in partman that cause partition tables to be marked dirty
  • Any partition table changes require the disk to be entirely unused during partitioning, which is infeasible in Ubiquity
    • Consider supporting this only for d-i

Resizable images

Add support for resizable virtual disks. This can be done via:

  • LVM
  • Use 4GB contiguous (zeroed) + nGB sparse
  • qcow2 type of images

MyDocuments and host bookamarks

Add bookmarks for MyDocuments and Host drive

  • Evan does not want to add this to m-a. He's afraid users would get horribly confused when they removed their NTFS partition and all of their documents suddenly disappeared. Since it's specific to Wubi, Evan thinks it should be done there.
  • It was agreed that bookmarks are a preferred solution over symlinks or bindmounts.
  • One complication of having a MyDocuments bookmark is that it does not necessarily lie in the /host volume, so that the partition might not be mounted and/or mounted with r/o permissions.

  • see xdg-user-dir

Sysvinit improvements

  • In umountfs unmount items in /proc/mounts in reverse order stopping at /.
  • Don suggested using lazy unmount (-l) but it should not be required given the above, also lazy unmounts do not help with /host since that contains / which is never unmounted.
  • Keep the -f option, but either fix its behaviour within umount when handling bindmounts (now umount -f bindmount will unmount the underlying device) or keep using WEAKMOUNTPOINTS

Porting Wubi to other platforms

  • Support installing inside a folder (no loopfile). Handy for Lubi and Mac ports.
  • very important that this be separated such that it imposes zero cost on most Ubuntu users
  • lubi (wubi for linux)
  • mubi (wubi for mac)

Porting Wubi To Debian

  • Requires porting upstream all the changes to initramfs-tools, grub & co, sysvinit scripts, casper/d-i initrd hooks, lupin-support.

    • ... and almost certainly most of Ubuntu's d-i customisations! This is a large project (which should be done anyway, but will take a while and contains some elements which are likely to be contentious).
    • console-setup needs to become the default in Debian
    • localechooser encoding support needs to be fixed
  • Investigate the opportunity of a tighter integration with the win32-loader project (at the moment it does netinstallation only).

Wishlist

  • Support for \"large\" distros such as Ubuntu Studio or Ubuntu Education Edition (support separate add-on CD directly? or leave it as post-installation task? shall we automate such tasks somehow?)
  • support DVD
  • Bootable USB device with ISO with casper-rw
  • Grub2 (as recommended by bean123)
    • grub is still unsupported in Ubuntu anyway, and it won't be critical to have grub4dos unsupported too
    • we should move over eventually, but consensus is \"not yet\"
  • Built-in CD burner (need to evaluate CD burning libraries)
  • Online resizing (in particular squeezing of ntfs partition while it is mounted and even hosting /)
  • native ntfs installations (last time I checked szaka mentioned it was 2 years away)
  • fallocate in ntfs/vfat (we can generate the virtual disk images via the windows frontend)

Brainstorm

Wubi in brainstorm

BoF agenda and discussion


CategorySpec

WubiJaunty (last edited 2008-12-11 20:25:00 by 216)