Its been a year since we've released Joomla! 1.5 and what surprises me is how little third party ETL plugins have been written for the migrator. When David Gal released the first migrator many developers complained that it wasn't configurable and that they can't be a part of the migration. So building a pluggable migrator was my main goal and the migrator for 1.0 was to be completely pluggable: you can add as many tables as you want, including third party migrator plugins. As far as I know only JoomlaTools wrote an ETL plugin for DocMAN with many third party developers suggesting that their users should manually backup and transfer their data. This is strange when writing an ETL plugin would mean that the data in the database is exported with the migrator and then imported into your new 1.5 site during install. Beyond installing the ETL plugin, there are no extra steps as part of the install except for having to install the extensions back on the live site.

So how hard is it to write an ETL plugin? JoomlaTools actually wrote a blog post on writing an ETL plugin in April last year and we've got some documentation on the Joomla! docs site as well. My old presentation dated August 2007 also has some information on writing an ETL plugin as well and the basic system hasnt changed much since then either. Basically you fill in some data about the name of the table you want to migrate a descriptive name and the SQL required to create the table so that it can load the distribution. It really is that simple if all you are doing is pulling your data straight across. But it can do so much more.

ETL stands for extraction, transformation and load. The extraction phase is what most people will use and if all you're doing is shifting the tables (without changing schema or fields) then there is no need to transform anything. The basic ETL plugin we document only does this with no transformation. The power with transformation is the ability to rename, remove or add fields to the table as well as change the content of those fields as the data exports out of your 1.0 site. Where this can be really handy for is if you're writing an ETL plugin for an old 1.0 extension that has been abandoned so that you can convert it to your own 1.5 native extensions data format. This way the transition from 1.0 and 1.5 is almost seemless except that you install the newer 1.5 version of the extension.

And to write an ETL plugin, you don't need to be the original developer of the extension as you probably guessed from the above. If you're an extension developer I strongly encourage you to write an ETL plugin to help your users transfer their data from their 1.0 site to 1.5 as well, however even if you're not the extensions developer you can still write an ETL plugin as well. I'd also encourage anyone who writes an ETL plugin to submit it to the ETL plugin section of the JED (JoomlaTools' DocMAN ETL Plugins are looking a bit lonely). The more ETL plugins we get out there, the happier everyone will be and the easier it will be to migrate from 1.0 to 1.5 :)