Back to blog home

Doing the heavy lifting: Gearman and Magento

With most web applications, there comes a time where there is a need to perform either CPU or I/O intensive work based on user actions. Whether processing uploaded files, or performing system-wide database updates, developers are increasingly turning to Gearman as a simple way to hand off the heavy lifting to another server to be performed asynchronously. It is easy to install and configure and has interfaces to many popular languages, including PHP.With this in mind, the team at Ibuildings created a module to allow easy usage of Gearman from within the open source ecommerce solution Magento. Releasing this module provides a simple way to perform job queue operations within the popular open-source ecommerce platform. The module is open source and freely available for download at This tutorial walks through installation of the module and some examples using it; it is assumed that you are already familiar with Magento development to begin this tutorial.


Once you have a working installation of Magento, you will need to install Gearman and the PHP classes to work with it. Here are some example steps, aimed at a Debian-based system for illustration purposes.

$ sudo apt-get install libgearman-dev
$ sudo pecl install gearman


While developing, you might wish to run the Gearman worker also on your development environment or virtual machine. If so, then you will need to install the server packages for Gearman as well: 

$ sudo apt-get install gearman-job-server


At this point you can install the Ibuildings_Gearman module into your Magento installation. Change into the {MAGENTO_ROOT}/app directory and install the module directly from GitHub with the following command:

$ git git://


Once you have cleared your cache, you should then go to the System > Configuration page, and look for the "Ibuildings Gearman Module" configuration section. Here you will need to add the IP address and port for your Gearman server. If you are running on a development machine with a local Gearman job server, the settings will be "" and "4730" respectively.


There were two scenarios that were envisaged when developing this component: a situation where you would "fire and forget" the job, where you can safely ignore the status of the job during its lifecycle, or where you would like a reference to the job so that you can check its status periodically and update the user as to job progress. For the first situation, the module employs the Magento Event/Observer pattern, and can be used from a Model object as follows:


$event = array();
$event['queue'] = 'test';
$event['task']  = array(
    'id'       => 1234,
    'data'     => 'This is a string!',
    'callback' => ''
Mage::dispatchEvent('gearman_do_async_task', $event);


Whilst the event is dispatched in the standard Magento way, the contents of the event array itself bear some explanation. The $event['queue'] element contains the name of the queue that you wish to send the job to. It is important that the name of this is correct as otherwise your jobs will either not be processed, or processed by another queue entirely! The $event['task'] element is what is passed to the Gearman worker for further processing and may contain any data relevant to the task performed. For this example, a job ID, some data and a callback URI are provided to the worker. The event dispatcher will take care of packaging this into a format that Gearman can work with. The second way of using the Gearman module involves instantiating the Queue object so that a reference to the Gearman job may be returned and used later for checking the job status. This is achieved as follows:

$task = array();
$task['queue']    = 'test';
$task['task']     = array(
    'id'         => 1234,
    'payload'    => 'This is a string!'
$queue = Mage::getModel('gearman/queue');
$id = $queue->dispatchTask($task);


This gives you the returned job ID so you can monitor the status for your task as and when required. This could be especially useful in conjunction with an Ajax call which polls the job status for the UI. However, should you wish to block the calling process until the work has been done, it would be possible by creating a loop such as this:

do {
    $ret = $queue->checkTaskComplete($id);
while (!$ret);


Obviously, having a web server process sitting and waiting for a response is not the optimal use for Gearman, however, it can be a good way to synchronise the processing of resources. Using this approach negates all of the benefits of having a job queue. However, I have included the example as it was useful to use when writing unit tests, where it is important to verify that the job does get processed. Workers are written in the standard Gearman way although you may wish to include Mage.php so that your existing models can be used when doing your background work. Below is a trivial example worker that calls a (non-Magento) static class method to perform the work.

$worker = new GearmanWorker();
$worker->addServer();    // Assumes localhost and default port
    array('MyStaticClass', 'workerMethod')
echo "Waiting for work...n";
while ($worker->work()) {
    if ($worker->returnCode() !== GEARMAN_SUCCESS) {
        echo "Ooops: " . $worker->returnCode() . "n";
class MyStaticClass
    public static function workerMethod($job)
        $task = unserialize($job->workload());
        $task['payload'] = strrev($task['payload']);
        if (isset($task['callback'])) {
            $task['result'] = file_get_contents(
        return serialize($task);


Whilst not specific to Gearman, the task optionally makes use of a callback URL to notify when the work has been done – this is of particular use when an event is fired and you have no reference to query your job status once it has been called.


Message queues are gaining prominence in the world of web applications, and rightly so. They vastly simplify the task of scaling your application and can dramatically increase responsiveness in your user interface. With this module, I hope that we have shown you that your Magento shop can also share in these benefits with very little impact on your existing codebase.


General documentation for Magento may be found at the Magento website. For resources relating to creating a Magento module, you can find an excellent article here on techPortal or on Alan Storm's blog.