# Upgrading From Previous Versions

## Upgrading from all 2.4 series to 2.5

- It is strongly advised to shut down the server completely while performing the upgrade, because newer daemons will write mailboxes.db entries which aren't compatible with the old daemons, and may cause crashes if you try to upgrade a running server. You can start the server again immediately with the new daemons installed.
- The most important change is that index file upgrades are no longer automatic. This avoids the IO storm caused by upgrades from 2.3 to 2.4. Even if you are upgrading from Cyrus 2.2, it won't cause an IO storm any more.
  Instead, you need to run `reconstruct -V max` to upgrade each mailbox in turn. This will upgrade all mailboxes to the current minor version 13. This can be safely done while the server is running.
- Older version mailboxes will continue to work "forever", though newer features like annotation quotas will fail, since there's no space in the mailbox header to store them.
  There may be slightly inconsistent search results until you upgrade. The cyrus.cache format in 2.3 and earlier ran words together in cache, so searches won't obey word boundaries like they do in 2.4+
- If you were running Cyrus 2.3.x with expunge_mode: delayed, then the contents of the cyrus.expunge file will be removed the first time the mailbox is opened, and the mailbox will act as if expunge_mode: immediate was set until it is upgraded. This is because version 2.3 mailboxes used the cyrus.expunge file, and it's not worth the complexity to try to recreate that file.
- After all mailboxes are upgraded, you should run `quota -f` to make sure all the message and folder count quotas are correct. Cyrus 2.5 supports new quota types, and the counts need to be calculated.
- Many configuration options have changed and new features have been added, read the release notes.
- Mailboxes.db and quota internal formats have changed.
- Just go read the release notes already: https://docs.cyrus.foundation/imap/release-notes/2.5.0.html

## Upgrading from 2.4.17-caldav-beta10 or earlier

- The schema used for the CalDAV database has changed. ALL sites MUST run the `dav_reconstruct` utility for each of their CalDAV users.

## Upgrading from 2.4.17-caldav-beta9 or earlier

- The time span calculations for recurring events in which the component containing the RRULE is not the first component have been fixed. We recommend ALL sites run the `dav_reconstruct` utility for each of their CalDAV users.

## Upgrading from 2.4.17-caldav-beta6 or earlier

- The time span calculations for various VCALENDAR components, especially VTODO (tasks), have been fixed. We recommend ALL sites run the `dav_reconstruct` utility for each of their CalDAV users.

## Upgrading from 2.4.12

- some LOG_DEBUG level messages are no longer created by default in the duplicate delivery system, so you won't see them in syslog any more
- mailboxes in the DELETED. hierarchy are now always suppressed for non-admin users, even if delayed delete is not enabled (see Bug #3604). If you happen to be using this namespace at your site for something else, you're crazy - but you can set "deletedprefix" in imapd.conf to an invalid value (e.g. '-') if you need.

## Upgrading from 2.4.11

- To support bug-interoperability with older versions of Cyrus, the quota command now supports "-1" as a synonym for unlimited storage

## Upgrading from 2.4.10

- Duplicate database formats have changed, so any duplicate or vacation data will be lost - meaning that vacation responses will be sent again, and duplicates just across the upgrade time will both get delivered.

## Upgrading from 2.4.9

- quota -f now works correctly. If you are upgrading from earlier than 2.4.9 we recommend that you run quota -f to ensure that all quotas are now correct. There is a slight race condition in the quota command, so it's safest to run with the server shut down, but this has always existed.
- On the topic of quota -f, the documentation has been updated to make it quite clear that you need to give "complete" prefixes if using quotalegacy, as it doesn't implement quite the same semantics as other databases. If you run 'quota -f a' and you have domains starting with 'a', you will be quite sad at the results. So don't do that. Running quota -f with no argument, or with a full domain or full user specification is perfectly safe.

## Upgrading from 2.4.8

- CYRUS_PREFIX environment varabile. If you have an environment variable "CYRUS_PREFIX" then config files will be searched for in there first, so for example the file /var/cyrus/etc/imapd.conf will override /etc/imapd.conf if your CYRUS_PREFIX is /var/cyrus. This is to make things easier for sites with multiple installs on a single machine. You will want to check that there aren't unexpected files in those locations when upgrading.
- New config options: *_db_path allow moving individual DB files to different partitions - for example you may want to put the deliver.db onto tmpfs to improve performance.
- KNOWN BUG: quota -f can double ALL quota usage values for all users. This is particularly bad because of the bug fixed in this release where quotaroots were not correctly updated on folder rename. Recommended workaround: run quota -f twice.

## Upgrading from 2.4.7

- New config option: `proc_path` allows setting the path to the proc directory onto tmpfs more easily

## Upgrading from 2.4.6

- New config option: `failedloginpause` allows you to change the pause after a failed login from the existing default of 3 seconds
- On Solaris: now getpassphrase is used, so passwords longer than 8 characters will work with management tools
- $confdir/proc now gets created automatically if it doesn't exist, which may impact init script design
- If you have damaged mailboxes that weren't previously fixable, a reconstruct of those mailboxes may be advised

## Upgrading from 2.4.5

- New config option: `suppress_capabilities`, which takes a space separated list of capabilities which will NOT be given in any imap capability response. This can be used on frontends to not display capabilities that older backends don't have, so clients don't get confused

## Upgrading from 2.4.4

- sync_client no longer forks two process - it just pre-forks a single process in rolling replication mode and uses it "forever". This is to avoid the bug where it used to open the BDB environment once before forking, then close it in every child, causing the reference count to go negative. This also means you can start it without '-o' and still have the master start without the replica already running.

## Upgrading from 2.4.3

- Actually, it's more upgrading directly to here from 2.3.x or older. ALL upgrades from previous versions now incur a full re-parse of all messages, which may take a while. In exchange you get reliable upgrades though, and guaranteed still-searchable cyrus.cache records.

## Upgrading from 2.4.2

- The AFS ptloader configure options have changed. If you were previously specifying --with-afs when you built, you'll now want to use --enable-afs instead. Additionally, you may now specify --with-afs-libdir and --with-afs-incdir to facilitate finding the AFS libraries and header files.
- The sync_log_names option has been replaced with sync_log_channels and the documentation and tools updated to match. This is to use a less overloaded term and allow multi-channel replication to be used.
- Cyrus 2.4.3 can now XFER mailboxes back to earlier version (at least as far back as Cyrus 2.2.12) in a murder configuration.
- sync_client in rolling mode now only connects after forking, so it no longer blocks startup. You can remove the '-o' option from your cyrus.conf entry if you want, and not have to start sync_client manually afterwards!

## Upgrading from 2.4.0

- The response to the "ID" command has changed slightly to include git metadata rather than CVS metadata - so your in-house tools expecting a particular value may need changing (very unlikely)

## Upgrading from 2.3.16

- There is a new index format, so downgrades are not possible without a reconstruct. We recommend that you take a full backup before upgrading, and perform an upgrade with a small test server if possible to make sure everything will work for your site.
- MURDER does work, but you can't rename a mailbox back to a previous version backend because the index was upgraded.
- There is a new replication protocol, so you can't replicate between 2.4.0 and previous versions in either direction. You need to upgrade both before you can restart replication.
- The default value for `delayed_expunge` has changed to provide better QRESYNC and replication support. The default is still pretty efficient, but you may want to change it to '`immediate`', the previous default
- The default type for all databases is now `skiplist` which is very reliable now, all the bugs are ironed out! Because ctl_cyrusdb -r automatically converts databases between known types, you shouldn't need to do anything, but if you want to keep the old defaults, you'll need to make them explicit in your imapd.conf as follows:

      duplicate_db: berkeley-nosync
      ptscache_db: berkeley
      statuscache_db: berkeley-nosync
      tlscache_db: berkeley-nosync

- There is a new lock folder which defaults to configdir/lock/ and contains one zerobyte file per mailbox. These can get pretty hot, and don't need to persist over reboots (they will be auto-created when needed) - so you may want to define `mboxname_lockpath` to be on tmpfs or ramfs or similar. It certainly makes sense to clean it out on restart, because names will persist in there forever otherwise. Even on mailbox delete these files aren't removed (to avoid potential race conditions)
- The make_md5 and make_sha1 utilities no longer exist. If you want to check message file integrity, the best way is reconstruct -r -n -G. This will parse every message file, check the sha1 against the value stored in the cyrus.index, and tell you about mismatches without fixing them.
- Speaking of reconstruct, the switches have changed, as has the implementation. It fixes the mailbox "in place" a lot more rather than creating a new mailbox, meaning that IMAP semantics won't be broken. It's more likely to abort rather than complete if your filesystem permissions are wrong - fix them first!

  -k | Removed, expunge data is always kept
  -g | Removed, GUIDs are always stored
  -G | Changed, now means "re-parse the message". If not specified, reconstruct only checks the message size and assumes the rest of the stuff in the index record is OK (pretty safe because we do CRC32 checks on index records now)
  -s | Added, skip the `stat` call, so reconstruct doesn't even check the message file size - just that a file with the correct name exists at all
  -n | Added, don't make any changes, just report what would have been done if -n wasn't specified

  If you try to use -k or -g, you will get a warning but the reconstruct will still run.

## Upgrading from 2.3.15

- The SQL detection code in configure has been reworked. Separate options for the include directories and lib directories have been added. Previous SQL options may not work as expected.

## Upgrading from 2.3.10

- STARTTLS is now allowed for externally preauth'd LMTP connections. If you don't want STARTTLS to be advertised and used by preauth'd clients, you can set `<service_name>_tls_cert_file: disabled` in imapd.conf.

## Upgrading from 2.3.9

- The method used for generating Globally Unique IDentifiers used for replication has been changed to be the SHA1 hash of the messages. If you wish to upgrade the existing GUIDs in particular mailbox(es) or the entire server, perform the following steps in the listed order. Note that it is NOT REQUIRED that existing GUIDs be upgraded.
  1. Zero GUIDs on the replica (`reconstruct -g`)
  2. Regenerate GUIDs on the master (`reconstruct -G`)
  3. Regenerate GUIDs on the replica (`reconstruct -G`)

## Upgrading from 2.3.8

- You must install and configure Cyrus SASL version 2.1.17 or later to use Cyrus IMAP 2.3.9 and later. You can download SASL at http://www.cyrusimap.org/mediawiki/index.php/Downloads#SASL_Library.
- The default value of the `allowplaintext` option has been changed to disabled (0). If you need to allow cleartext passwords on the wire, then you will have to explicitly enable the `allowplaintext` option in `imapd.conf`.

## Upgrading from 2.3.3 or later (64-bit machines)

- Due to byte alignment issues in cyrus.index, all mailboxes will have to be reconstructed.

## Upgrading from 2.3.4 or 2.3.5

- Any mailboxes which had messages appended/delivered/copied with a 2.3.4 service or copied with a 2.3.5 imapd MUST be reconstructed in order for the new messages to be displayed by clients.

## Upgrading from 2.2.x or earlier

- If you wish to use separate metadata partition(s), you MUST first shut down Cyrus and then perform the following:
  1. Set the `metapartition-*` and `metapartition_files` options to suit your configuration. For a full description of these options, see the `imapd.conf(5)` man page.
  2. Create the metadata partition directory(s) listed in the `metapartition-*` option(s), setting the ownership and permissions in same fashion as step 6 of install-configure.
  3. Run the `tools/migrate-metadata` script (as the cyrus user) to move the metadata files listed in the `metapartition_files` option from the spool partition(s) to the new metadata partition(s). This script may take a long time to run depending on the number of mailboxes on the server, but presumably the metadata partitions are located on high speed storage, so the writes should be relatively fast.
  4. Restart Cyrus.

## Upgrading from 2.2.2 or earlier

- The Cyrus database backend configuration is now handled at runtime using `imapd.conf` options. If you are not using the default backend for any of the databases, make sure that you specify the correct backend(s) in appropriate option(s).
- The format of the `newspeer` option has been changed. The existing format will still be parsed, but the option should be upgraded to use the new format (see `imapd.conf(5)` for details).

## Upgrading from 2.2.1 or earlier

- The sieve bytecode format has changed again to correct an issue with the short circuiting of the allof and anyof operators. To upgrade existing scripts (outside of home directories), you can run the `tools/masssievec` perl script included with the distribution. It requires a path to your `sievec` binary. This should also upgrade scripts that have already been compiled to bytecode. For example:

      masssievec /usr/src/cyrus/sieve/sievec

## Upgrading from 2.2.0 or earlier

- The improved directory hashing (fulldirhash) is now a runtime configuration option. If you are currently using this feature, then make sure that you enable the `fulldirhash` option in `imapd.conf`.
- The format of mailbox index files has changed. They are upgraded on the fly, so you need to do nothing to upgrade. However, to downgrade them you will need to remove the cyrus.index files, and reconstruct the mailboxes, otherwise the index files will be invalid.
- `ctl_deliver -E` has been deprecated in favor of `cyr_expire -E`. This new tool does both duplicate delivery database pruning as well as message expunging. You should replace the appropriate `EVENTS` entry in `cyrus.conf` with one of those in the sample configurations in the `master/conf` directory.
- The sieve bytecode format has changed. The new format is encoded in network byte order, and will be transferable between architectures. To upgrade existing scripts (outside of home directories), you can run the `tools/masssievec` perl script included with the distribution. It requires a path to your `sievec` binary. This should also upgrade scripts that have already been compiled to bytecode. For example:

      masssievec /usr/src/cyrus/sieve/sievec

## Upgrading from 2.1.x or earlier

### General information (ALL SITES)

- The default database formats for the mailbox list and the seen state databases has been changed to the skiplist backend. There are two ways of dealing with this if you have been using the defaults.
  1. Specify `--with-mboxlist-db=berkeley` and `--with-seen-db=flat` to `configure`. This will instruct Cyrus to continue to use the previous defaults.
  2. Use the `cvt_cyrusdb` program to directly convert the databases. This should be done with the server down, and with the binaries from the new Cyrus distribution. Change any paths that do not match your configuration.
     For the mailbox list, the command looks like:

         /usr/cyrus/bin/cvt_cyrusdb /var/imap/mailboxes.db berkeley /var/imap/mailboxes.db.new skiplist
         mv /var/imap/mailboxes.db.new /var/imap/mailboxes.db

     Note that the use of full paths to the database files is important. You should also backup your old mailboxes database before moving the new one in.
     For the seen state databases, the command to get them all in one fell swoop looks like:

         find /var/imap/user -name \*.seen -exec /usr/cyrus/bin/cvt_cyrusdb \{\} flat \{\}.new skiplist \; -exec mv \{\}.new \{\} \;

     The slashes are important for shell escaping. Again, you should back up the contents of your `/var/imap/user` directory before executing this command. These commands may take some time to complete, especially if your databases are large.
  We believe that skiplist offers considerable performance advantages for these two databases over the previous defaults.
- Sieve scripts are now compiled into bytecode. The program `sievec` is provided to do this process manually (timsieved will compile submitted sieve scripts as they are uploaded). To upgrade existing scripts (outside of home directories), you can run the `tools/masssievec` perl script included with the distribution. It requires a path to your `sievec` binary. For example:

      masssievec /usr/src/cyrus/sieve/sievec

  Note that this will fail for scripts that use the "envelope" extension but do not require it. Cyrus 2.1's `timsieved` did not do appropriate checking that the optional envelope test was required before it was used.
- Configuration subsystem changes:
  - The tls_[service]_* configuration options have been removed. Now use [servicename]_tls_*, where servicename is the service identifier from `cyrus.conf` for that particular process.
  - The `admins` and `lmtp_admins` configuration options no longer union. Per-service options completely override the default value when they are specified.
  - `lmtp_allowplaintext` is no longer a defined parameter and must be specified using the service name of your lmtp process if you require a specific value.

### Specialized information (Murder, AFS, etc.)

- The IMAP IDLE command is now supported by proxyd and is controlled by the `imapidlepoll` option, which is enabled by default (60 seconds). To disable IMAP IDLE in proxyd, set `imapidlepoll` to 0.
- User moves via RENAME and XFER are now controlled by the `allowusermoves` option, which defaults to off.
- If you use `ptloader`, it now runs as a regular cyrus service. This means that you will need master to acquire and maintain AFS tokens for it. You will also need to create the ptclient directory under your imap configdirectory, to hold the PTS cache (now a full-fledged cyrusdb) and UNIX socket. In `cyrus.conf`, ptloader should be setup to listen on <configdirectory>/ptclient/ptsock. See the `master/test/cmu-backend.conf` example configuration file.
- Also, `ptloader` has been given a generic interface. You should now specify "`--with-auth=pts`" (instead of "`--with-auth=krb_pts`") to `configure`. There is also a `--with-pts=` `configure` option that defaults to `afskrb` (Kerberos Canonicalization, AFS PTS Groups). There is also an experimental ldap module. Note also that if `ptloader` fails the lookup, authorization (and therefore authentication) will now fail, as canonicalization is done inside of ptloader.
- The format of sieve referrals has changed to be more consistent with the current managesieve draft, this may cause interoperability problems when using managesieve clients and servers from different cyrus versions.
- Clients that use old-style ACL commands that include the "MAILBOX" directive will no longer function. We do not know of any clients that have this problem currently.
- Any applications that link libcyrus.a now need to link libcyrus_min.a as well.

## Upgrading from 2.1.13 or earlier

- We are now more forgiving of MIME boundary headers generated by earlier versions of eudora. However, if you have messages already in the mailstore that you want to fix you will need to reconstruct the affected mailboxes to regenerate the cached bodystructure data to take this into account. Nothing needs to be done for new messages to be treated in this way.

## Upgrading from 2.1.12 or earlier

- timsieved was corrected to behave properly in the altnamespace configuration. However, this means that it was previously looking for sieve scripts in "user.name" format instead of the (correct) "user^name" format. A sample script to do this (which should be run in the top level of the sieve directory) is in `tools/convert-sieve.pl`. Note that this is only needed if you are running with altnamespace turned on.

## Upgrading from 2.1.3 or earlier

- If you use notifications (previously `notify_zephyr` or `notify_unix`) this functionality has been separated out to `notifyd`. See the `notifyd` manpage and example entries in `master/conf`.

## Upgrading from 2.1.2 or earlier

- Sieve has been updated to be compliant with RFC 3028 and draft-martin-sieve-notify-01. All `notify` actions and any `fileinto` and/or `redirect` actions using stringlists will have to be updated/changed.

## Upgrading from 2.0.16 or earlier

- You must install and configure Cyrus SASL version 2 to use Cyrus IMAP 2.1 and later. You can download SASL at http://www.cyrusimap.org/mediawiki/index.php/Downloads#SASL_Library.
- If you use `timsieved` to manage Sieve scripts, and have enabled the alternate namespace and/or the Unix hierarchy separator, run the script "`tools/translatesieve`". This script will translate the folder names in `fileinto` actions.
- Cyrus now uses the service name "sieve" instead of "imap" for the SASL profile of `timsieved`. If you use `timsieved` to manage Sieve scripts, be sure to update your password checking mechanism appropriately,
- If you have enabled the improved directory hashing scheme, run the script "`tools/rehash full`". This script will rehash your existing directories.
- The hashed deliver databases (used for duplicate delivery suppression and Sieve) have been merged into a single `deliver.db` database. You can safely remove the entire `/var/imap/deliverdb` directory structure after shutting down the server.
- All of the Cyrus databases have been unified under a single BDB environment. A new `ctl_cyrusdb` tool is now used for database recovery and checkpointing instead of `ctl_mboxlist` and `ctl_deliver`. You should replace the appropriate `START` and `EVENTS` entries in `cyrus.conf` with those in the sample configurations in the `master/conf` directory.
- Cyrus now caches SSL/TLS sessions in an external database. If you have support for SSL/TLS, and haven't disabled session caching (see `imapd.conf(5)`), you should add a line like the following to the `EVENTS` section of `cyrus.conf` to prune expired sessions from the database:

         # this is only necessary if caching TLS sessions
         tlsprune      cmd="tls_prune" period=1440

## Upgrading from 2.0.6, 2.0.7, 2.0.8, or 2.0.9 or earlier

- If you use `timsieved` to manage Sieve scripts, run the script "`tools/upgradesieve`". `timsieved` now uses symlinks instead of hard links.

## Upgrading from a previous 2.0 version to 2.0.6

Warning: You do not need to follow these instructions if you're upgrading from version 1.6.

- You can now pick whether to use Berkeley db to store seen state, the subscription files, and the mailboxes file or a flat text file, at compile time only. (Look in `imap/seen_db.c` and `imap/mboxlist.h`.)
- The format of the mailboxes file and seen state has changed. It is not possible to preserve seen state, but upgrade the mailboxes file as follows:
  1. Run `ctl_mboxlist -d > mboxlist.temp` to dump existing mailboxes.
  2. Remove old database files: `rm mailboxes.db db/* user/*/*.seen`
  3. With the new version of ctl_mboxlist, run `ctl_mboxlist -u < mboxlist.temp`.

## Upgrading from 1.6.22 or 1.6.24

Warning: Cyrus imapd 2.0 will automatically convert on-disk file formats as the server is used. It is not possible to run 1.6 after 2.0 has been used on a mail spool without reconstructing every mailbox.

- Create some extra directories and remove the duplicate delivery database:

         mkdir /var/imap/db
         mkdir /var/imap/socket
         chown cyrus /var/imap/db /var/imap/socket
         rm -rf /var/imap/deliverdb

- Convert mailboxes file to Berkeley DB:

         su cyrus
         cd /var/imap
         ctl_mboxlist -u < mailboxes
         ctl_cyrusdb -c

  Please keep a backup of your mailboxes file. You can dump an old-style mailboxes file by using `ctl_mboxlist -d`.
- remove "`/etc/inetd.conf`" entries. The `imap` and `popd3d` lines need to be removed from `/etc/inetd.conf` and inetd needs to be restarted.
- master process configuration: You'll need to configure the master process Cyrus process and ensure that it starts on boot. see this section of the configuration instructions.
- MTA configuration. You will have to reconfigure your MTA to speak to lmtpd. See this section of the configuration document.
- cyrus.seen conversion. The cyrus.seen file will be automatically upgraded as users read mail. After some time, you might want to delete the cyrus.seen file in each mailbox; it is superseded by the user/joe.seen file.
- cyrus.index conversion. The cyrus.index file will be automatically upgraded the first time each mailbox is SELECTed.
- Netnews conversion. The netnews programs are no longer built. If you are using netnews, you will need to apply the diff in the `netnews/` directory to INN or see if INN is now distributing those changes. You will also want to run `remotepurge` on a regular basis to purge old netnews posts.

## Upgrading from 1.6.13

- Upgrading from the Cyrus IMAP server version 1.6.13 or earlier: if you use Sieve, you should run the "`tools/upgradesieve`" script, as the format of the "`/usr/sieve`" directory has changed slightly.

  timsieved, included in this release, will handle maintenance of Sieve scripts.

- Upgrading from the Cyrus IMAP server version 1.6.10 or earlier: if you export news via the IMAP server, you'll have to change your "`newsfeeds`" file to contain

      collectnews!:*:Tf,WR:collectnews

  The format of the input to collectnews has changed.

  Duplicate delivery suppression is now required for Sieve.

- Upgrading from the Cyrus IMAP server version 1.6.1 or earlier (including 1.5.x!): the quota and user directories are now hashed by the first character of the username. This is to reduce the number of entries in any given directory. It doesn't do a great job (and in some cases it will do a really poor job) but as a quick hack it shouldn't make things worse. Optionally, the data partitions can also be hashed by enabling the "hashimapspool" option.

  You must hash your directories using the "`dohash`" script in the tools subdirectory. (If you want to hash your mail spool, be sure to set "hashimapspool" before running "`dohash`".) This must be run as the Cyrus user. Be sure to stop mail service while converting. Doing this in single user mode is probably the safest.

## Upgrading from 1.5

- Upgrading from the Cyrus IMAP server version 1.5 or earlier: libsasl is now required. Configuring SASL to work may be a chore, especially if you use shadow passwords.
- An ANSI C compiler is now required. gcc should work fine and can be acquired from http://www.gnu.org/software/gcc/gcc.html.
- Make sure to read the upgrading instructions under 1.6 above.
- Upgrading from 1.5.14 or earlier requires deleting the delivered database. Remove the file delivered.db in the configdirectory and make a directory called "deliverdb" in the configdirectory. This may cause some duplicates to get through.
- Upgrading from 1.5.14 or earlier requires removing the PTS cache database (if the AFS PTS group support is used, which is not the default). The PTS cache is in /var/ptclient/ptscache.db, and you should remove it. This is because the format for the PTS cache for IMSP has changed. If you use AFS ACLs, IMSPd, and IMAPd on the same machine, make sure you have version 1.5a5 of the IMSP server for this version of the IMAP server. (If you don't have IMSP, or AFS, don't worry about it.)

last modified: $Date: 2010/01/06 17:01:29 $