Akka.NET v1.5 Upgrade Advisories

This document contains specific upgrade suggestions, warnings, and notices that you will want to pay attention to when upgrading between versions within the Akka.NET v1.5 roadmap.

Upgrading to Akka.NET v1.5.32

Future Breaking Change in Akka.Cluster.Tools

The method ClusterSingleton.Init() will be removed in future v1.6, if you're using this method, you need to convert it to use ClusterSingletonManager.Props and ClusterSingletonProxy.Props instead.

In order to preserve backward compatibility within your cluster, if you're using this convention:

var settings = ClusterSingletonSettings.Create(system);
var singletonActor = SingletonActor.Create(Counter.Props, "GlobalCounter")
    .WithStopMessage(MyStopMessage.Instance)
    .WithSettings(settings);
var proxy = singleton.Init(singletonActor);

You will need to convert it to:

var managerSettings = ClusterSingletonManagerSettings.Create(system)
    .WithSingletonName("GlobalCounter");
system.ActorOf(
    props: ClusterSingletonManager.Props(
        singletonProps: Counter.Props,
        terminationMessage: MyStopMessage.Instance,
        settings: managerSettings),
    name: "singletonManagerGlobalCounter");

var proxySettings = ClusterSingletonProxySettings.Create(system)
    .WithSingletonName("GlobalCounter");
var proxy = system.ActorOf(
    props: ClusterSingletonProxy.Props(
        singletonManagerPath: "/user/singletonManagerGlobalCounter",
        settings: proxySettings),
    name: "singletonProxyGlobalCounter");

Note that to preserve backward compatibility between cluster nodes, the singleton manager actor name MUST be in the $"singletonManager{singletonName}" format.

Upgrading to Akka.NET v1.5.31

Akka.NET v1.5.31 introduces a breaking behavior change to actor Stash. In previous behavior, Stash will filter out any messages that are identical (see explanation below) when it is prepended with another. It will not do so now, which is the actual intended behavior.

This change will affect Akka.Persistence users or users who use the Stash.Prepend() method in their code. You will need to add a de-duplication code if your code depends on sending identical messages multiple times to a persistence actor while it is recovering.

Messages are considered as identical if they are sent from the same sender actor and have a payload message that Equals() to true against another message. Example payload types would be an object pointing to the same memory address (ReferenceEquals() returns true), value types (enum, primitives, structs), and classes that implements the IEquatable interface.

Upgrading to Akka.NET v1.5.15

Akka.NET v1.5.15 introduces several major changes:

Akka.Analyzers

The core Akka NuGet package now references Akka.Analyzers, a new set of Roslyn Code Analysis and Code Fix Providers that we distribute via NuGet. You can see the full set of supported Akka.Analyzers rules here.

Akka.Cluster.Sharding Changes

In #6863 we made some major changes to the Akka.Cluster.Sharding API aimed at helping improve Cluster.Sharding's performance and ease of use. However, these changes may require some effort on the part of the end user in order to take full advantage:

  • ExtractEntityId and ExtractShardId have been deprecated as they fundamentally can't be extended and can't benefit from the performance improvements introduced into Akka.NET v1.5.15. It is imperative that you migrate to using the HashCodeMessageExtractor instead.
  • You no longer need to handle ShardRegion.StartEntity or ShardingEnvelope inside your IMessageExtractor implementations, and in fact AK2001 (part of Akka.Analyzers) will automatically detect this and remove those handlers for you. Akka.NET automatically handles these two message types internally now.

ClusterClient Serialization Changes

In #7032 we solved a long-standing serialization problem with the ClusterClient where Send, SendToAll, and Publish were not handled by the correct internal serializer. This has been fixed by default in Akka.NET v1.5.15, but this can potentially cause wire compatibility problems during upgrades - therefore we have introduced a configuration setting to toggle this:

# re-enable legacy serialization
akka.cluster.client.use-legacy-serialization = on

That setting is currently set to on by default, so v1.5.15 will still behave like previous versions of Akka.NET. However, if you have been affected by serialization issues with the ClusterClient (such as #6803) you should toggle this setting to off.

Upgrading to Akka.NET v1.5.2

Akka.NET v1.5.2 introduces two important behavioral changes:

We meant to include both of these changes in Akka.NET v1.5.0 but simply ran out of time before making them into that release.

Akka.Persistence Changes

The impact of Akka.Persistence: need to remove hard-coded Newtonsoft.Json object serializer is pretty minor: all versions of Akka.NET prior to 1.5.2 used Newtonsoft.Json as the object serializer for Akka.Persistence regardless of whether or not you used a custom object serializer, such as Hyperion.

Going forward your user-defined object serialization binding will now be respected by Akka.Persistence. Any old data previously saved using Newtonsoft.Json will continue to be recovered automatically by Newtonsoft.Json - it's only the serialization of new objects inserted after upgrading to v1.5.2 that will be affected.

If you never changed your object serializer (most users don't) then this change doesn't affect you.

Akka.Cluster Split Brain Resolver Changes

As of Akka.NET v1.5.2 we've now enabled the keep-majority Split Brain Resolver by default.

If you were already running with a custom SBR enabled, this change won't affect you.

If you weren't running with an SBR enabled, you should read the Akka.Cluster Split Brain Resolver documentation.

Also worth noting: we've deprecated the akka.cluster.auto-down-unreachable-after setting as it's always been a poor and shoddy way to manage network partitions inside Akka.Cluster. If you have that setting enabled you'll see the following warning appear:

The `auto-down-unreachable-after` feature has been deprecated as of Akka.NET v1.5.2 and will be removed in a future version of Akka.NET.
The `keep-majority` split brain resolver will be used instead. See https://getakka.net/articles/cluster/split-brain-resolver.html for more details.

Disabling the Default Downing Provider

To disable the default Akka.Cluster downing provider, simply configure the following in your HOCON:

akka.cluster.downing-provider-class = ""

This will disable the split brain resolver / downing provider functionality altogether in Akka.NET. This was the default behavior for Akka.Cluster as of Akka.NET v1.5.1 and earlier.

Upgrading From Akka.NET v1.4 to v1.5

In case you need help upgrading:

But first: review this document!

Akka.Cluster.Sharding State Storage

One of the most significant upgrades we've made in Akka.NET v1.5 is a complete rewrite of Akka.Cluster.Sharding's state storage system.

Note

You can watch our discussion of this Akka.Cluster.Sharding upgrade during our September, 2022 Akka.NET Community Standup for more details.

In Akka.NET v1.5 we've split Akka.Cluster.Sharding's state-store-mode into two parts:

  • CoordinatorStore (akka.cluster.sharding.state-store-mode) and
  • ShardStore (akka.cluster.sharding.remember-entities-store.)

Which can use different persistence modes configured via akka.cluster.sharding.state-store-mode & akka.cluster.sharding.remember-entities-store.

Important

The goal behind this split was to remove the ShardCoordinator as a single point of bottleneck during remember-entities=on recovery - in Akka.NET v1.4 all remember-entity state is concentrated in the journal of the PersistentShardCoordinator or the CRDT used by the DDataCoordinator. In v1.5 the responsibility for remembering entities has been pushed to the Shard actors themselves, which allows for remembered-entities to be recovered in parallel for all shards.

Possible combinations:

state-store-mode remember-entities-store CoordinatorStore mode ShardStore mode
persistence (default) - (ignored) persistence persistence
ddata ddata ddata ddata
ddata eventsourced (new) ddata persistence

There should be no breaking changes from user perspective. Only some internal messages/objects were moved. There should be no change in the PersistentId behavior and default persistent configuration (akka.cluster.sharding.state-store-mode)

This change is designed to speed up the performance of Akka.Cluster.Sharding coordinator recovery by moving remember-entities recovery into separate actors - this also solves major performance problems with the ddata recovery mode overall.

The recommended settings for maximum ease-of-use for Akka.Cluster.Sharding in new applications going forward will be:

akka.cluster.sharding{
  state-store-mode = ddata
  remember-entities-store = eventsourced
}

However, for the sake of backwards compatibility the Akka.Cluster.Sharding defaults have been left as-is:

akka.cluster.sharding{
  state-store-mode = persistence
  # remember-entities-store (not set - also uses legacy Akka.Persistence)
}
Warning

state-store-mode=persistence will be deprecated and state-store-mode=ddata will eventually be made the default. Make plans to migrate off of persistence urgently.

Migrating to DData From Akka.Persistence with Remember Entities

This section applies to users who are using remember-entities=on and want to migrate to using the low-latency event-sourced based storage. All other users should just migrate to state-store-mode=ddata.

Switching over to using remember-entities-store = eventsourced will cause an initial migration of data from the ShardCoordinator's journal into separate event journals going forward.

Upgrading to Akka.NET v1.5 will cause an irreversible migration of Akka.Cluster.Sharding data for users who were previously running akka.cluster.state-store-mode=persistence, so follow the steps below carefully:

Important

This migration is intended to be performed via upgrading Akka.NET to v1.5 and applying the recommended configuration changes below - it will require a full restart of your cluster any time you change the state-store-mode setting.

Update your Akka.Cluster.Sharding HOCON to look like the following (adjust as necessary for your custom settings):

akka.cluster.sharding {
    remember-entities = on
    remember-entities-store = eventsourced
    state-store-mode = ddata

    # fail if upgrade doesn't succeed
    fail-on-invalid-entity-state-transition = on
}

 akka.persistence.journal.{your-journal-implementation} {
    event-adapters {
        coordinator-migration = ""Akka.Cluster.Sharding.OldCoordinatorStateMigrationEventAdapter, Akka.Cluster.Sharding""
    }

    event-adapter-bindings {
        ""Akka.Cluster.Sharding.ShardCoordinator+IDomainEvent, Akka.Cluster.Sharding"" = coordinator-migration
    }
}

Migrating to DData From Akka.Persistence without Remember Entities

If you're migrating from state-store-mode=persistence to state-store-mode=ddata and don't use remember-entities=on, then all you have to configure is the following:

akka.cluster.sharding {
    state-store-mode = ddata
}

Executing Migration

Important

This section only applies to users migrating from state-store-mode=persistence to state-store-mode=ddata. For all other users there is no need to plan a special deployment - the transition to Akka.NET v1.5 sharding will be seamless. It's only the changing of state-store-mode settings that requires a restart of the cluster.

To deploy this upgrade:

  1. Take your cluster offline and
  2. Roll out the changes with the new version of Akka.NET installed and these HOCON changes.

It should less than 10 seconds to fully migrate over to the new format and the Akka.Cluster.Sharding system will continue to start normally while it takes place.

Note

If you don't run Akka.Cluster.Sharding with remember-entities=on normally then there is no need to turn it on here.

With these HOCON settings in-place the following will happen:

  1. The old PersitentShardCoordinator state will be broken up - remember-entities=on data will be distributed to each of the PersistentShard actors, who will now use the new remember-entities-store = "eventsourced" setting going forward;
  2. Old Akka.Cluster.Sharding.ShardCoordinator+IDomainEvent will be upgraded to a new storage format via the coordinator-migration Akka.Persistence event adapter; and
  3. No more data will be persisted by the ShardCoordinator - instead it will all be replicated on the fly by DData, which is vastly preferable.
Important

This migration is irreversible once completed.

If you run into any trouble upgrading, please file an issue with Akka.NET.

Breaking Logging Changes

In v1.5, we've re-engineering the ILoggingAdapter construct to be more extensible and performant. Unfortunately this necessitate some breaking changes that will affect end-user code - but the remedy for those changes is trivial.

After installing the v1.5 NuGet packages into your applications or libraries, you will need to add the following to all of your source files where you previously made calls to the ILoggingAdapter:

using Akka.Event;

That using statement will pull in the extension methods that match all of the v1.4 API ILoggingAdapter signatures in v1.5.

Even better - if you can take advantage of global using statements in C#10, then this is a one-liner as either the MSBuild or project level:

In Directory.Build.props:

<Project>
    <ItemGroup>
        <PackageReference Include="Akka" />
        <Using Include="Akka.Event" />
    </ItemGroup>
</Project>

Note, in order to log more than 6 parameters you need to explicitly create object[] and pass your arguments into corresponding method. It was done in order to eliminate large numbers of accidental object[] allocations that were created implicitly through the params keyword AND to eliminate the large amount of boxing that occurred whenever value types (struct) were logged:

var @params = new[]{arg1, arg2, arg3, arg4, arg5, arg6, arg7};
loggingAdapter.Log(format, @params);