Projections
The XOOM LATTICE way to project state, Domain Events, and other Source types into a CQRS Query Model.
Projections are an important part of the component set required to support CQRS, whether or not you are using Event Sourcing. Recall that in CQRS there are two models, one called the Command Model and another called the Query Model.
The Command Model is often referred to as the "Write Model" and the Query Model as the "Read Model." These AKAs are quite inaccurate because you actually read from and write to both models. We prefer the names Command Model and Query Model because these adhere to the pattern name, CQRS, and express the purpose of each of the two models.
A projection is a component that takes one set of states and transforms those into another state. This is typically how it works:
The Command Model causes a state mutation and emits one or more Domain Events
The new Domain Events are persisted into a
Journal
The emitted Domain Events are dispatched/routed to a set of Projection components based on their interest in a given Domain Event type
The Projection components interpret how each Domain Event should be reflected into the portion of the Query Model for which they are responsible, and project that interpretation into the persistent Query Model
In all of the above steps, you can replace "Domain Event" with "entity state," because you may decide not to use Domain Events in your Command Model. In other words, the full state of a Command Model entity may be interpreted for projection into the Query Model. Of course if you use Event Sourcing you will always project Domain Events into the Query Model.
With the XOOM LATTICE projection components, we provide the means to project an entity's state and various concrete Source<?>
types—DomainEvent
and Command
, for example—into a CQRS query model.
There are two steps required to use projections. You must implement one or more projections and you must configure those projections to be used within your service/application. These two topics are covered below.
Projection Set Up and Configuration
The components used to manage the data filtering, routing, and the actual projections, must be created. The following are the component types.
The
ProjectToDescription
is a description of how data and events are filtered and selected. You may configure aProjectToDescription
for projectingState<T>
and also for projectingSource<T>
instances, such asDomainEvent
types. Both are described below.The
Dispatcher
andProjectionDispatcher
to be registered with the storage mechanism is created by means of an actor that implements both protocols. Two such are theBinaryProjectionDispatcherActor
andTextProjectionDispatcherActor
, and used when a binary store or text store is used, respectively.The store is created and
Dispatcher
registered with it.
The following describes each of the above steps.
ProjectToDescription
for State<T>
ProjectToDescription
for State<T>
A ProjectToDescription
may be defined for state data and for events. The following shows a description defined for a state object.
State data may have associated Metadata
, and the Metadata
instance may have a named operation
that caused the state to transition to the current value. The above description indicates that if the state data contains an operation
that matches "User:new"
, "User:contact"
, or "User:name"
, the dispatcher will route the data to the UserProjectionActor
.
The following demonstrates how the operation
name is associated with the new state. In this case the command is to replace the user's name with a different name. The transitioned state and the descriptive operation is applied. After the state is persisted, the Dispatcher
is given the opportunity to route the new state to a projection. As seen in the description of the previous code snippet, matching the "User:name"
operation will cause the new state to be routed to the UserProjectionActor
.
We provide a comprehensive example of using this full-state projection technique, known as the vlingo-http-frontservice.
ProjectToDescription
for Source<T>
ProjectToDescription
for Source<T>
A ProjectToDescription
may be defined for state data and for events. The following shows a description defined for events.
The above description indicates that if a concrete DomainEvent
is one of the types ForumStarted
, ForumClosed
, or ForumModeratorAssigned
, the dispatcher will route the event to the ForumProjectionActor
.
The following demonstrates how the ForumStarted
event occurs. After the event is persisted, the Dispatcher
is given the opportunity to route it to a projection. As seen in the description of the previous code snippet, matching the ForumStarted
event will cause it to be routed to the ForumProjectionActor
.
Note however that keeping the ProjectToDescription
up to date with new and changing event types could be error prone by being overlooked. Instead of referencing each event type, you may instead reference the Java package that contains all of the events.
Dispatcher
and ProjectionDispatcher
Dispatcher
and ProjectionDispatcher
The following demonstrates how the ProjectToDescription
, Dispatcher
, ProjectionDispatcher
, TextProjectionDispatcherActor
, and Journal
are wired for use.
You may want to have an isolated component to configure your projections, such as this ProjectionDispatcherProvider
example.
There are three primary objects created.
Notable are the three
ProjectToDescription
instances created, which are formed into a singleList<ProjectToDescription>
. Each of the descriptions indicate how various command model changes will be dispatched into the actor instance whose type is the first parameter. For example, thisDashboardProjectionActor
instance will be created inside theProjectionDispatcher
and allDomainEvent
types defined as inner classes of thecom.dashdawn.model.dashboard.Events
class, will be dispatched to it. The same goes for theWaveBoardDetailsProjectionActor
and the innerEvents
types defined in its package.The
descriptions
are then used to create theProjectionDispatcher
, which is implemented by theTextProjectionDispatcherActor
provided by XOOM LATTICE. Actually this actor implements two protocols, both theDispatcher
and theProjectionDispatcher
. TheDispatcher
is used by theJournal
to dispatch newly appended events to theProjectionDispatcher
. (This is implemented by the abstract baseProjectionDispatcherActor
, which is extended by the concrete actors,TextProjectionDispatcherActor
andBinaryProjectionDispatcherActor
). The two protocol references answered byactorFor()
are captured in theProtocols
type.Lastly the
ProjectionDispatcherProvider
instance is created, where both protocol references are held and available to dependents.
Configuring the Dispatcher
The following is an example of how the Dispatcher
is used, where the command model storage will use it to dispatch events into the projections. Note that the command model store knows nothing about projections, only the Dispatcher
.
The above code may be placed in your Bootstrap
or Startup
class containing a main()
.
Implementing With the Projection
Interface
Projection
InterfaceThe following is an example of a Projection
designed to handle state mutation operations. It corresponds to the above section ProjectToDescription
for State<T>
.
The Projectable
contains the data to be projected. The ProjectionControl
is used to confirm that the Projectable
has been projected so that the Dispatcher
will not route it again. The code inside each of the cases would create or update the views effected by the state mutations.
The full example of the above projection approach is in the xoom-http-frontservice
.
The following is an example of a Projection
designed to handle events. This example corresponds to the above section ProjectToDescription
for Source<T>
.
Since there may be one or more events that were applied, persisted, dispatched, and routed to the projection, the projection must iterate over all entries so as not to miss any. The code inside each of the cases would create or update the views effected by the events. A much cleaner approach follows.
Implementing With the StateStoreProjectionActor
StateStoreProjectionActor
There is an Actor
abstract base type for this: StateStoreProjectionActor
. This is a much more powerful abstract than directly using the Projection
interface. As the name indicates, this projection base is used to project into a StateStore
.
Since object-relational mapping and other object persistence approaches are less efficient than key-value stores, we have decided not to support ORM-based projections at this time.
Here is the a condensed code listing of the StateStoreProjectionActor
.
In the following documentation we explain a number of available overridable base class methods. Note that these are all optional, except that one of the two merge(...)
methods should be overridden.
Override the following method if you are performing full-state projections:
protected T merge(
final T previousData,
final int previousVersion,
final T currentData,
final int currentVersion)
Override the following method if you are performing projections for Event Sourcing:
protected T merge(
final T previousData,
final int previousVersion,
final T currentData,
final int currentVersion,
final List> sources)
The default implementation behaviors of all non-merge methods will very likely serve your needs fully.
To create a new type of projection, extend StateStoreProjectionActor
, as demonstrated next. This example overrides the merge(...)
that is best used with Event Sourcing.
This projection is next examined in detail, starting with the constructor.
The DashboardProjectionActor
constructor uses the super constructor that takes only the StateStore
used to write and read DashboardView
instances. This single-parameter constructor provides a default StateAdapter
and EntryAdapter
for the concrete projection to use. These are part of XOOM SYMBIO and are known as DefaultTextStateAdapter
and DefaultTextEntryAdapter
.
There is a second constructor for StateStoreProjectionActor
that takes an additional two parameters, one for the StateAdapter
and one for the EntryAdapter
.
These adapters are used to adapt from persistent State
and Entry
types to the model types. When the single argument constructor is used you get the default StateAdapter
and EntryAdapter
.
Since there is only one EntryAdapter
supported, but no doubt several or many DomainEvent
types needed, your own custom EntryAdapter
must internally hold the full number of EntryAdapter
instances needed for all DomainEvent
types. This might use a Map
with keys of Class<? extends DomainEvent>
and values of concrete EntryAdapter
instances. The outer EntryAdapter
registered with the StateStoreProjectionActor
would operate by looking up the specific concrete EntryAdapter
by DomainEvent
type, dispatch to it, and answer its return value.
The DashboardProjectionActor
implements the Projection
protocol, but there is a default implementation of method projectWith()
provided by the abstract base classStateStoreProjectionActor
. The implementation of this method is quite simple for most uses. If you need a more involved implementation you may override the default provided by the abstract base class.
The simple step is to "upsert" (i.e. insert or update, depending on existence) to current Projectable
, which may hold a state, or entries, or both.
The upsertFor()
does the following.
Asks
currentDataFor(projectable)
for its notion of the current state, as in the state that is now being projected. You may overridecurrentDataFor(projectable)
. This may or may not be useful, but if it is, you must return the value that is considered current from theProjectable
. This method is generally used only for full-state projections rather than those based onDomainEvent
instances.Requests that the projection
prepareForMergeWith()
. The default behavior adapts anyentries()
in theProjectable
toSource<?>
instances, holding them in an internalList<Source<?>>
. The concrete extender may get the adaptedSource<?>
instances using thesources()
method. These are provided by default with the secondmerge(...)
method that is used for Event Sourcing projections. If you have some alternative or additional preparation to do before the merge this method must be overridden.Reads the current
T
value from theStateStore
(see the last statement in the method) and passes aBiConsumer<T, Integer> upserter
. When the read completes, theupserter
is used to merge thepreviousData
with thecurrentData
and/or apply anySource<?>
instances.Writes the merged state back to the
StateStore
ifalwaysWrite()
istrue
or if the newly merged data is not equal to the previous data. This happens inside theupserter
.
The DashboardProjectionActor
implements currentDataFor()
as follows.
As previously indicated, the currentDataFor()
may or may not be useful in terms of providing a new state. This implementation answers an Empty
instance of the DashboardView
, which will be used by the merge()
as an accumulator of modifications. This particular override example is unnecessary but demonstrates an alternative to the default implementation. Assume that we decide not to implement the above override and remove the Empty
value declaration as well as the currentDataFor()
method override.
If you are not projecting from events, but the full state of the domain object, an implementation of currentDataFor()
and merge()
similar to the previous section is necessary.
Next the DashboardProjectionActor
is given the opportunity to prepare for the merge.
In this implementation the prepareForMergeWith()
adapts all Entry
instances into instances of the DomainEvent
specialization IdentifiedDomainEvent
. This gives the concrete event types the means to provide the identity of the event instance. As indicated above, this specific implementation of prepareForMergeWith()
is redundant because it does the same that the default behavior already does. You would choose to override prepareForMergeWith()
only if you required behavior beyond the above implementation.
Immediately following prepareForMergeWith()
the upsert()
calls dataIdFor()
.
As seen above, the dataId
is provided by the first (and possibly only) instance of the IdentifiedDomainEvent
in the events
list. Recall that this events
list was just previously populated by the prepareForMergeWith()
. This example dataIdFor()
method override is likely redundant because the default behavior provides a more thorough implementation, first checking for the Projectable dataId()
, and if that is not available, looks for the identity from one of the first IdentifiedDomainEvent
. If neither of these are available then the dataId
will be a blank String
, and an override of this method would be necessary to provide a custom value.
If you are using the default implementation of prepareForMergeWith()
, the source/event instances are retrieved as follows. Here again, your DomainEvent
instances must be of the extends IdentifiedDomainEvent
type, which can provide the identity()
of the entity that emitted them.
The following is the DashboardProjectionActor
implementation of merge()
and its helper method mergeInto()
, assuming that the prepareForMergeWith()
was also overridden and a local List<DomainEvent>
of events
was collected by it. This example shows an alternative to using the more appropriate merge(...)
override that provides the List<Source<?>>
parameter.
The merge()
passes the currentData
to be used by mergeInto()
as an accumulator of the new state. Any preexisting state on the currentData
that coincides with the interpretation of the DomainEvent
will be overwritten/replaced.
Because the Projectable
may have multiple entries (serialized DomainEvent
instances), the mergeInto()
iterates over the entire collection of events deserialized by the method prepareForMergeWith()
.
Assuming you are using the default prepareForMergeWith()
implementation, you would iterate as follows, using the sources()
method rather than a collection that you gathered on your own.
The switch
uses an enum
of DashboardViewProjectableType
to match on each of the DomainEvent
instances.
The DomainEvent
type is matched to the enum by the match()
method. A non-match produces the Unmatched
type, which is ignored by mergeInto()
, but logged as a warning.
Once a specific type is matched, the mergeInto()
maps event attributes to DashboardView
attributes. Any preexisting values that are mapped are overwritten on the view
accumulator. The iteration and matching is done for any number of events, which may be included with the current Projectable
or others received in the future.
Note that the design of the DashboardView
to directly use public mutable attributes was done to keep the implementation simple, albeit somewhat "dangerous" according to opinions. It doesn't seem like a bad thing in this specific case because the view object is used in very specific limited ways. Feel free to include accessor methods for reading and writing in your designs.
When the merge()
completes, the modified DashboardView
is written to the StateStore
.
Specialized Data Versions and Updates
The StateStoreProjectionActor<T>
provides a default means to determine the current version of the data to be merged and then updated to the database. It works like this:
In other words, by default the data version provided by the Projectable
is used as the version for the current data to be merged and updated to the database. This is how all of the above examples obtain the current data version for inserts/updates. It is based on a policy to always update the current merged data. That is, the data is always written no matter whether it is the same as the previous data, or different.
Yet, in some cases you may want to use a different versioning policy and also conditionally write the newly merged data instance only if it is different from the previous data. This is useful for idempotent operations, such as when the Projectable
is redelivered due to a latent or failed delivery confirmation. In such cases the StateStoreProjectionActor<T>
provides a secondary policy, which is to automatically increment the previous version by one and to also test the current merged data with the previous data to qualify the need to update the data to the database.
To use the secondary policy, you should override the alwaysWrite()
method. By default it returns true
. Your override should return false
instead.
The second method involved in the policy is currentDataVersionFor()
, which you need not override if the secondary policy works for your specific projection use case.
As you can see, the currentDataVersionFor()
provides both policies in one line of code. If alwaysWrite()
answers true
, the projectable.dataVersion()
is returned as was explained above. Otherwise, when alwaysWrite()
answers false
, the next sequential version from the one previously stored in the databased is returned. Note that in the case where there was no previous data in the database for the given identity, the previousVersion
is -1
. Thus, there is the one special case where 1
must be explicitly returned as the initial version.
As previously indicated, using the secondary version and write policy is useful when your use case must be idempotent. Yet, there is another case to consider. If you are projecting multiple entity states and/or events into a single projection data instance, the versions from the various instances will almost certainly collide with each other.
For example, ProductDefined
from one Product
entity will have version 1
, and ProductDefined
from a different Product
entity will also have version 1
. This is unavoidable because ProductDefined
is always the first event emitted from a Product
. If you attempt to merge the ProductDefined
that is delivered after the first initially delivered ProductDefined
, the merged/updated data will have version 1
just as the previous data did. Writing this version will cause an optimistic concurrency violation in the StateStore
, because the version will make the new data look as if it is based on stale state. By overriding alwaysWrite()
to answer false
, your current/merged data version will always be one greater than the previous data version.
But what about the idempotent check? In the case where alwaysWrite()
answers false
, there will be a pre-update check requiring the current data to be different from the previous data.
Notice specifically this statement.
The data difference check is managed by using the equals()
of the data's type. The updating write is performed only when the current and previous data are not equal. Note that the equals()
implementation must check for value equality (compare type equality as well as equality of all corresponding data attributes/fields) rather than entity equality (compare ids only).
Last updated