Guzzle provides a fairly generic and very customizable batching framework that allows developers to efficiently transfer requests in parallel.
You can send HTTP requests in parallel by passing an array of Guzzle\Http\Message\RequestInterface
objects to
Guzzle\Http\Client::send()
:
$responses = $client->send(array(
$client->get('http://www.example.com/foo'),
$client->get('http://www.example.com/baz')
$client->get('http://www.example.com/bar')
));
You can send commands in parallel by passing an array of Guzzle\Service\Command\CommandInterface
objects
Guzzle\Service\Client::execute()
:
$commands = $client->execute(array(
$client->getCommand('foo'),
$client->getCommand('baz'),
$client->getCommand('bar')
));
These approaches work well for most use-cases. When you need more control over the requests that are sent in
parallel or you need to send a large number of requests, you need to use the functionality provided in the
Guzzle\Batch
namespace.
The batch object, Guzzle\Batch\Batch
, is a queue. You add requests to the queue until you are ready to transfer
all of the requests. In order to efficiently transfer the items in the queue, the batch object delegates the
responsibility of dividing the queue into manageable parts to a divisor (Guzzle\Batch\BatchDivisorInterface
).
The batch object then iterates over each array of items created by the divisor and sends them to the batch object's
Guzzle\Batch\BatchTransferInterface
.
use Guzzle\Batch\Batch;
use Guzzle\Http\BatchRequestTransfer;
// BatchRequestTransfer acts as both the divisor and transfer strategy
$transferStrategy = new BatchRequestTransfer(10);
$divisorStrategy = $transferStrategy;
$batch = new Batch($transferStrategy, $divisorStrategy);
// Add some requests to the batch queue
$batch->add($request1)
->add($request2)
->add($request3);
// Flush the queue and retrieve the flushed items
$arrayOfTransferredRequests = $batch->flush();
Note
You might find that your transfer strategy will need to act as both the divisor and transfer strategy.
The Guzzle\Batch\BatchBuilder
makes it easier to create batch objects. The batch builder also provides an easier
way to add additional behaviors to your batch object.
The Guzzle\Http\BatchRequestTransfer
class efficiently transfers HTTP requests in parallel by grouping batches of
requests by the curl_multi handle that is used to transfer the requests.
use Guzzle\Batch\BatchBuilder;
$batch = BatchBuilder::factory()
->transferRequests(10)
->build();
The Guzzle\Service\Command\BatchCommandTransfer
class efficiently transfers service commands by grouping commands
by the client that is used to transfer them. You can add commands to a batch object that are transferred by different
clients, and the batch will handle the rest.
use Guzzle\Batch\BatchBuilder;
$batch = BatchBuilder::factory()
->transferCommands(10)
->build();
$batch->add($client->getCommand('foo'))
->add($client->getCommand('baz'))
->add($client->getCommand('bar'));
$commands = $batch->flush();
You can add various behaviors to your batch that allow for more customizable transfers.
Use the Guzzle\Batch\FlushingBatch
decorator when you want to pump a large number of items into a batch queue and
have the queue automatically flush when the size of the queue reaches a certain threshold.
use Guzzle\Batch\BatchBuilder;
$batch = BatchBuilder::factory()
->transferRequests(10)
->autoFlushAt(10)
->build();
Batch builder method: autoFlushAt($threshold)
Use the Guzzle\Batch\NotifyingBatch
decorator if you want a function to be notified each time the batch queue is
flushed. This is useful when paired with the flushing batch decorator. Pass a callable to the notify()
method of
a batch builder to use this decorator with the builder.
use Guzzle\Batch\BatchBuilder;
$batch = BatchBuilder::factory()
->transferRequests(10)
->autoFlushAt(10)
->notify(function (array $transferredItems) {
echo 'Transferred ' . count($transferredItems) . "items\n";
})
->build();
Batch builder method:: notify(callable $callback)
Use the Guzzle\Batch\HistoryBatch
decorator if you want to maintain a history of all the items transferred with
the batch queue.
use Guzzle\Batch\BatchBuilder;
$batch = BatchBuilder::factory()
->transferRequests(10)
->keepHistory()
->build();
After transferring items, you can use the getHistory()
of a batch to retrieve an array of transferred items. Be
sure to periodically clear the history using clearHistory()
.
Batch builder method: keepHistory()
Use the Guzzle\Batch\ExceptionBufferingBatch
decorator to buffer exceptions during a transfer so that you can
transfer as many items as possible then deal with the errored batches after the transfer completes. After transfer,
use the getExceptions()
method of a batch to retrieve an array of
Guzzle\Batch\Exception\BatchTransferException
objects. You can use these exceptions to attempt to retry the
failed batches. Be sure to clear the buffered exceptions when you are done with them by using the
clearExceptions()
method.
Batch builder method: bufferExceptions()