Migrate/Upgrade Ubuntu 14.04 LTS (Trusty Tahr) GNU/Linux from 32 bit to 64 bit HowTo

1. Introduction

This brief HOWTO shows how to take a working 32 bit 14.04 installation and convert it into a 64 bit system. You will be able to keep all your programs, settings and data. You start with a 32 bit system that can run 64 bit programs, then convert it to a 64 bit system that can run 32 bit programs.

With the release of Debian GNU/Linux wheezy, a new feature called multiarch was introduced. The facility permits the installation of executable files (such as libraries and programs) from multiple architectures simultaneously on the same system. The primary use of this is to allow 32 bit programs to be run on a 64 bit system, but it has other uses.

With the release of Ubuntu LTS 14.04 (Trusty Tahr), the ability to use the multiarch facility to upgrade to 64 bit is easier than ever. Much simpler than upgrading 12.04 and an evolution of upgrading Debian 7 or 8.

If you are using Ubuntu LTS 18.04 (Bionic Beaver) the process is quite different so please use the dedicated guide on the newer page instead of this guide.

2. Preparation

Before we change any packages, preserve the list of the packages we have installed. At the end we can run a comparison or reconciliation to see if there are any omissions or remaining work. All steps should be performed using a VT text console rather than the GUI because the GUI will become unavailable during parts of the process. In addition, the steps should be run as root directly, avoiding the use of sudo. Please refer to this comment for further details about this.

dpkg --get-selections "*:i386"|grep -v deinstall >dpkg-get-selections.cp.00.i386
dpkg --get-selections "*:amd64"|grep -v deinstall >dpkg-get-selections.cp.00.amd64
dpkg --get-selections "*:all"|grep -v deinstall >dpkg-get-selections.cp.00.all

3. Backups

Have I mentioned the importance of backups? Ensure you have a tested recoverable backup of your entire system, and also a means to access it should the network or kernel be unavailable. If you have encrypted files or home directories then verify that you can restore these successfully.

Some packages store their data in a binary format that differs between architectures. We must grab the data in an architecture-independent manner now, in order to restore it in the 64 bit system later. Such packages I have installed are OpenLDAP and PostgreSQL.

3.1. OpenLDAP

Dump the OpenLDAP database and preserve its configuration. We'll have to purge the package as it will attempt and fail to upgrade its database upon migration.

umask 77
slapcat -n 1 >slapcat-n1
service slapd stop
mkdir -p etc/ldap
rm -Rf etc/ldap/slapd.d
cp -ai /etc/ldap/slapd.d etc/ldap

3.2 PostgreSQL

Dump the PostgreSQL cluster data and preserve its configuration.

su - postgres -c "umask 77; pg_dumpall >/tmp/pg_dumpall_9_3.sql"
mv /tmp/pg_dumpall_9_3.sql .
bzip2 -f pg_dumpall_9_3.sql
mkdir -p etc/postgresql/9.3/main
cp -pi /etc/postgresql/9.3/main/pg_hba.conf etc/postgresql/9.3/main

4. Boot into a 64 bit kernel

The 32 bit kernel does not allow the execution of 64 bit programs. However the 64 bit kernel allows the execution of both 32 bit and 64 bit programs, so this 64 bit kernel is the first thing needed. Run the following in turn to allow 64 bit programs to be installed as an additional architecture, then install the 64 bit kernel. We will also install debootstrap and we will use it later to download the base system.

The steps below will replace the 32 bit kernel with the 64 bit kernel. If you do not have any other 32 bit kernels installed then I recommend backing up the current kernel and initrd before running the commands.

dpkg --add-architecture amd64
apt-get update
apt-get install linux-generic:amd64 debootstrap

Now reboot with the newly installed 64 bit amd64 kernel before continuing.

5. Prepare base systems

Cross-grading the whole existing system is prone to error due to the mass of dependencies, tied with the fact that dpkg currently cannot cross-grade (change architecture) and upgrade (change version) a package in the same operation.

Download the base system's essential packages in 32 bit and 64 bit varieties. The less we download here, the quicker the base system migration will be as we will need to cross-grade them all from 32 bit to 64 bit. If you don't require networking then do not include openssh-client, openssh-server, network-manager and systemd-shim in the --include list below. If you do not require the kernel, which is very slow to migrate, do not include linux-generic and grub-pc. If you don't need LVM then do not include lvm2, base-files and bash.

mkdir -p bootstrap-i386
rm -Rf bootstrap-i386/*
debootstrap --arch=i386 --download-only --include=openssh-client,openssh-server,network-manager,systemd-shim,linux-generic,grub-pc,lvm2,base-files,bash trusty ./bootstrap-i386

mkdir -p bootstrap-amd64
rm -Rf bootstrap-amd64/*
debootstrap --arch=amd64 --download-only --include=openssh-client,openssh-server,network-manager,systemd-shim,linux-generic,grub-pc,lvm2,base-files,bash trusty ./bootstrap-amd64

Now build a package list of the intended stripped-down base system.

ls -1 bootstrap-amd64/var/cache/apt/archives/|egrep "\.deb$"|cut -d "_" -f1|awk '{print $1 " install"}' >dpkg-get-selections.base

6. Purge architecture-independent packages

Purge any packages that cannot be migrated to 64 bit directly. I also recommend removing non-standard package repositories and removing (or preferably purging) any packages installed from those now.

apt-get purge slapd
apt-get purge postgresql-9.3
apt-get remove myspell-en-au # due to missing dependency on dictionaries-common

7. Strip to 32 bit base system

Remove all non-essential packages from the system. Note that rebooting during this process is not recommended as the system will be unbootable until the process has completed and the 64 bit kernel reinstalled.

Update our package list from the base system prepared earlier.

dpkg --clear-selections
dpkg --set-selections <dpkg-get-selections.base

Now remove all the non-essential packages that are not part of the base system.  This will take some time to complete.

apt-get dselect-upgrade

Downgrade any packages to the versions downloaded by debootstrap. This will strip updates and security fixes. We must have the same versions of packages in the 32 bit and 64 bit base system to be able to cross-grade them. They will of course be upgraded again later.

dpkg -i bootstrap-i386/var/cache/apt/archives/*.deb

Finally, install (migrate to) the 64 bit base system. To avoid complex dependency calculations, cross-grade the packages multiple times until all dependencies have been met. If we chose a small base system to download with debootstrap earlier then this step will be significantly faster.

dpkg --clear-selections
RC=1; while [ $RC -ne 0 ]; do
  dpkg --abort-after=1000 -i bootstrap-amd64/var/cache/apt/archives/*.deb
  RC=$?
done

Now the 64 bit kernel has been reinstalled (okay, probably an older one), so if you need to reboot, you could do so from this point onwards.

dpkg -l "linux-image-*"

Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name           Version      Architecture Description
+++-==============-============-============-=================================
un  linux-image-3.               (no description available)
ii  linux-image-3. 3.13.0-24.46 amd64        Linux kernel image for version 3.
rc  linux-image-3. 3.13.0-29.53 amd64        Linux kernel image for version 3.
ii  linux-image-ex 3.13.0-24.46 amd64        Linux kernel extra modules for ve
rc  linux-image-ex 3.13.0-29.53 amd64        Linux kernel extra modules for ve
ii  linux-image-ge 3.13.0.24.28 amd64        Generic Linux kernel image

Checkpoint the list of installed packages and then upgrade the base system.

dpkg --get-selections "*:i386"|grep -v deinstall >dpkg-get-selections.cp.01.i386
dpkg --get-selections "*:amd64"|grep -v deinstall >dpkg-get-selections.cp.01.amd64
dpkg --get-selections "*:all"|grep -v deinstall >dpkg-get-selections.cp.01.all

apt-get autoremove
apt-get dselect-upgrade

If you excluded the kernel from the above installation steps, then install it again now. This step will also ensure that all dependencies are met. Capture another package list checkpoint at the end of this process in case we need to refer back to it.

apt-get install locales linux-generic grub-pc lvm2

dpkg --get-selections "*:i386"|grep -v deinstall >dpkg-get-selections.cp.02.i386
dpkg --get-selections "*:amd64"|grep -v deinstall >dpkg-get-selections.cp.02.amd64
dpkg --get-selections "*:all"|grep -v deinstall >dpkg-get-selections.cp.02.all

8. Install virtual packages (optional)

To speed up the installation progress of the remaining 64 bit packages you may wish to manually install some of them now, especially the larger ones. Performing this step will also allow apt to take care of the dependencies for us.

Before proceeding, note the following error and its resolution. Any package (typically library packages) could raise this error during the 64 bit installation.

Unpacking libsensors4:amd64 (1:3.3.4-2ubuntu1) ...
dpkg: error processing archive /var/cache/apt/archives/libsensors4_1%3a3.3.4-2ubuntu1_amd64.deb (--install):
 trying to overwrite shared '/etc/sensors.d/.placeholder', which is different from other instances of package libsensors4:amd64
Errors were encountered while processing:
 /var/cache/apt/archives/libsensors4_1%3a3.3.4-2ubuntu1_amd64.deb

If the 32 bit and 64 bit versions of the package are not able to co-exist you will need to purge the 32 bit package (and manually reinstall the 64 bit package if you hit the error above). For example.

dpkg --purge libsensors4

(Reading database ... 100613 files and directories currently installed.)
Removing libsensors4:i386 (1:3.3.4-2ubuntu1) ...
Purging configuration files for libsensors4:i386 (1:3.3.4-2ubuntu1) ...

dpkg -i /var/cache/apt/archives/libsensors4_1%3a3.3.4-2ubuntu1_amd64.deb

(Reading database ... 100615 files and directories currently installed.)
Preparing to unpack .../libsensors4_1%3a3.3.4-2ubuntu1_amd64.deb ...
Unpacking libsensors4:amd64 (1:3.3.4-2ubuntu1) ...
Setting up libsensors4:amd64 (1:3.3.4-2ubuntu1) ...
Processing triggers for libc-bin (2.19-0ubuntu6) ...

dpkg --configure -a

With this in mind, purge the 32 bit packages now that are known to cause issues with cross-grading to 64 bit. Then install the major pseudo-packages. I also had gnome-shell and texlive installed. Installing them now will ease the effort later in the process.

dpkg --purge libldap-2.4-2:i386 libpython2.7-minimal:i386 libgtk-3-0 libsensors4 libpulse0 libgl1-mesa-dri libqtcore4 libsane sni-qt libgnomevfs2-extra

apt-get install ubuntu-desktop ubuntu-minimal ubuntu-standard gnome-shell texlive

When configuring gdm I was given a choice of which windows manager to use. I chose the default lightdm.

Now that the desktop packages are installed you may wish to restart the display manager.

service lightdm restart

9. Install remaining packages

Now we can calculate the list of packages outstanding for reinstallation and install them. Firstly purge all library packages as they are no longer required and their 32 bit configurations may not be able to co-exist with their 64 bit replacements (as detailed earlier).

apt-get install dselect aptitude
dselect update
aptitude keep-all
aptitude forget-new
aptitude purge '~c^lib'

Build the final package list of all the packages we wish installed (i.e. the 64 bit versions of the original 32 bit packages and all of the previously-installed architecture-independent packages). Then we can finally remove the old 32 bit architecture.

cat dpkg-get-selections.cp.00.all|grep -v deinstall >dpkg-get-selections.final
cat dpkg-get-selections.cp.00.i386|grep -v ":i386"|grep -v deinstall|cut -f1|awk '{print $1 " install"}' >>dpkg-get-selections.final
cat dpkg-get-selections.cp.00.i386|grep ":i386"|grep -v deinstall|cut -d: -f1|awk '{print $1 " install"}' >>dpkg-get-selections.final
cat dpkg-get-selections.cp.00.amd64|grep ^linux-image|grep -v deinstall|cut -d: -f1|awk '{print $1 " install"}' >>dpkg-get-selections.final

dpkg --clear-selections
dpkg --remove-architecture i386
dselect update
cat dpkg-get-selections.final|dpkg --set-selections

We need to use various strategies to get all the packages we want installed. As yet I have been unable to determine the precise reason for this and orginally thought it was caused by this feature. When running the below dselect-upgrade you may see packages marked for removal. For example I found that apache2:i386 was in dpkg's available cache, but apache2:amd64 was not and saw an error such as this when attempting to set it as a dpkg selection. It was in the available file so I could not get to the bottom of this behavior. I would see an error such as this.

dpkg: warning: package not in database at line 876: apache2:amd64

We will resolve this along the way because if we don't then we will start pulling back in 32 bit packages.

Purge any known problematic packages that will not cross-grade to 64 bit and begin the first installation.

dpkg --purge openjdk-6-jre-headless openjdk-7-jre-headless
apt-get dselect-upgrade

Keep trying to install the remaining packages and clean up any dependency issues as we go. Apt will triumph over the earlier issues with dpkg and install the missing packages (such as apache2). I found an issue with flashplugin-installer as I had forgotten to disable the partner repository before starting.

apt-get install `dpkg --get-selections "*:i386"|grep -v deinstall|cut -f1|cut -d: -f1|sed ':a;N;$!ba;s/\n/ /g'`
apt-get install `dpkg --get-selections "*:all"|grep -v deinstall|cut -f1|cut -d: -f1|sed ':a;N;$!ba;s/\n/ /g'`
apt-get purge flashplugin-installer
apt-get install `dpkg --get-selections "*:amd64"|grep -v deinstall|cut -f1|cut -d: -f1|sed ':a;N;$!ba;s/\n/ /g'`

dpkg --purge freeglut3:i386 odbcinst1debian2:i386 phonon:i386
apt-get install `dpkg -l "*:i386"|egrep "^rc"|cut -d" " -f3| cut -d: -f1|sed ':a;N;$!ba;s/\n/ /g'`

Finally install the kernel metapackage and cancel all pending actions in aptitude (should it still be installed).

apt-get install linux-generic
aptitude keep-all

10. Restore backups

10.1. PostgreSQL

Restore the PostgreSQL configuration and database cluster data.

cp -p etc/postgresql/9.3/main/pg_hba.conf /etc/postgresql/9.3/main
service postgresql reload
bzcat pg_dumpall_9_3.sql.bz2 >/tmp/pg_dumpall_9_3.sql
su - postgres -c "cd /tmp; psql <pg_dumpall_9_3.sql >psql.out 2>psql.err"
mv /tmp/psql.* .
cat psql.err
rm /tmp/pg_dumpall_9_3.sql

10.2. OpenLDAP

Restore the OpenLDAP configuration and database.

service slapd stop
rm /var/lib/ldap/*
rm -R /etc/ldap/slapd.d
cp -a etc/ldap/slapd.d /etc/ldap
slapadd -n1 <slapcat-n1
chown -R openldap:openldap /var/lib/ldap/*
service slapd start

11. Final checks

Re-enable any third party package repositories and install any packages required from these.

Grab a final checkpoint of the packages we have installed. Compare the latest list to the original package list. There should not be anything missing now except perhaps packages that are not available for 64 bit that were available for 32 bit.

dpkg --get-selections "*:i386"|grep -v deinstall >dpkg-get-selections.cp.03.i386
dpkg --get-selections "*:amd64"|grep -v deinstall >dpkg-get-selections.cp.03.amd64
dpkg --get-selections "*:all"|grep -v deinstall >dpkg-get-selections.cp.03.all

sdiff -s dpkg-get-selections.cp.00.i386 dpkg-get-selections.cp.03.amd64|grep "<"

12. Install 32 bit packages (optional)

If you still want to install those old 32 bit packages, then enable the 32 bit architecture and install them as normal.

dpkg --add-architecture i386
apt-get update
apt-get install googleearth
...