Updated CMU published

Posted by: mstauber Category: General

An updated CMU Migration Ultility has been published, which contains significant improvements.

Hi all,

One of the more complex pieces of code we are maintaining is CMU - our Cobalt Migration Utility. It had fallen into a slightly deplorable state. It was still mostly working, but while doing so it was groaning, moaning and bitching. No more of that!

The following fixes and improvements were made:

Speed:

We all know a CMU-Export (and import) can take its sweet time. Walzing through all those tarballs? That's slow. Especially if there is lots of data. The tarballs CMU generates for Vsites and Users are regular TAR archives packed with Gnuzip. TAR is relatively fast, but the Gnuzip part is what takes the longest.

For that reason we now switched CMU to use the command line utility 'pigz' to compress the tarballs. Mind you: This is still in Gnuzip format. So it can be unpacked with regular Gnuzip and is 1:1 compatible. It's just that 'pigz' is multiprocessor capable and will use all available CPU cores for parallel processing of the compressed archive.

The speed increase this provides naturally depends on a lot of factors. Like how many CPU cores your server has available and also how good (or bad) your files can be compressed.

But I've seen speed increases between 5-10 times for a regular CMU-Export during testing. \o/

The CMU-Import now also goes considerably faster than before, as it also uses 'pigz' when unpacking the tarballs.

Bugfixes:

  • The warning message "Subroutine Compress::Zlib::gzFile::gzseek redefined" is now gone.
  • Helptexts for cmuExport and cmuImport (available via "-h" parameter) have been cleaned up and clarified.
  • Incorrect helptext for cmuImport parameter "-s" corrected. It is now "import server-admins (Resellers)" as it should be.
  • Various quota related issues improved and fixed. In the past you would sometimes get some weird characters on the screen if a cmuImport did run into quota issues. This will no longer happen.
  • Various other fixes.

Summary:

Both a cmuExport and a cmuImport should now be a much faster and much cleaner experience than before.

There are still a few identified and open issues, which we will address in future CMU updates. Such as the current inability to cmuImport subdomains. Another room for improvement is the direct export/import of DNS and MySQL via CMU, which will be added in the close future. As is both DNS and MySQL must be exported/imported separately via the methods explained in our Migration Guide.

The updated CMU is available as YUM update for all versions of BlueOnyx.


Return
General
Jun 8, 2016 Category: General Posted by: mstauber
Previous page: Development Next page: Mailing List