Sharded Daemon Process

Warning

This module is currently marked as may change because it is a new feature that needs feedback from real usage before finalizing the API. This means that API or semantics can change without warning or deprecation period. It is also not recommended to use this module in production just yet.

Introduction

Sharded Daemon Process provides a way to run N actors, each given a numeric id starting from 0 that are then kept alive and balanced across the cluster. When a re-balance is needed the actor is stopped and, triggered by a keep alive running on all nodes, started on a new node (the keep alive should be seen as an implementation detail and may change in future versions).

The intended use case is for splitting data processing workloads across a set number of workers that each get to work on a subset of the data that needs to be processed. This is commonly needed to create projections based on the event streams available from all the Persistent Actors in a CQRS application. Events are tagged with one out of N tags used to split the workload of consuming and updating a projection between N workers.

For cases where a single actor needs to be kept alive see Cluster Singleton

Basic Example

To set up a set of actors running with Sharded Daemon process each node in the cluster needs to run the same initialization when starting up:

var tags = new[] { "tag-1", "tag-2", "tag-3" };
ShardedDaemonProcess.Get(Sys).Init("TagProcessors", tags.Length, id => TagProcessor.Props(tags[id]));

Scalability

This cluster tool is intended for small numbers of consumers and will not scale well to a large set. In large clusters it is recommended to limit the nodes the sharded daemon process will run on using a role.

Push-Based Communication

ShardedDaemonProcess also supports push-based communication not too dissimilar from a round-robin Router:

// start the daemon process on the host
var name = "daemonTest";
var targetRole = "workers";
var numWorkers = 10;
var settings = ShardedDaemonProcessSettings.Create(Sys).WithRole(targetRole);
IActorRef host = ShardedDaemonProcess.Get(Sys).Init(name, numWorkers, EchoActor.EchoProps, settings, PoisonPill.Instance);

// ping some of the workers via the host
for(var i = 0; i < numWorkers; i++)
{
    var result = await host.Ask<int>(i);
    result.Should().Be(i);
}

The ShardedDaemonProcess.Init call returns an IActorRef - any messages you send to this IActorRef will be routed to one of the n worker instances you specified.

Daemon Proxies

You can also interact with ShardedDaemonProcess on nodes that don't use the same Akka.Cluster role as the ShardedDaemonProcess host itself.

// start the proxy on the proxy system, which runs on a different role not capable of hosting workers
IActorRef proxy = ShardedDaemonProcess.Get(_proxySystem).InitProxy(name, numWorkers, targetRole);

// ping some of the workers via the proxy
for(var i = 0; i < numWorkers; i++)
{
    var result = await proxy.Ask<int>(i);
    result.Should().Be(i);
}

Under the covers we use a ShardRegionProxy to forward any messages you send to the IActorRef returned by the ShardedDaemonProcess.InitProxy method.