Docs: Nodes

Tue, 27 Jul 2021 00:00:00 +0000

All nodes in a Stroom cluster must be configured correctly for them to communicate with each other.

Configuring Nodes

Open Monitoring/Nodes from the top menu. The nodes screen looks like this:

TODO

Screenshot

You need to edit each line by selecting it and then clicking the edit icon at the bottom. The URL for each node needs to be set as above but obviously substituting in the host name of the individual node, e.g. http://<HOST_NAME>:8080/stroom/clustercall.rpc

Nodes are expected to communicate with each other on port 8080 over http. Ensure you have configured your firewall to allow nodes to talk to each other over this port. You can configure the URL to use a different port and possibly HTTPS but performance will be better with HTTP as no SSL termination is required.

Once you have set the URLs of each node you should also set the master assignment priority for each node to be different to all of the others. In the image above the priorities have been set in a random fashion to ensure that node3 assumes the role of master node for as long as it is enabled. You also need to check all of the nodes are enabled that you want to take part in processing or any other jobs.

Keep refreshing the table until all nodes show healthy pings as above. If you do not get ping results for each node then they are not configured correctly.

Once a cluster is configured correctly you will get proper distribution of processing tasks and search will be able to access all nodes to take part in a distributed query.

Docs: Node Groups and Processing Profiles

Thu, 13 Aug 2026 00:00:00 +0000

By default a Processor filter Processor filter A Processor Filter is used to used to find Streams to process through the Pipeline associated with the Processor Filter. A Processor Filter consists of an expression tree to select which Streams to process and a tracker to track the what Streams have been processed.Click to see more details... may be processed by any node in the cluster, at any time, up to the limits set on the filter itself. Node Groups and Processing Profiles let you be more selective than that.

A Node Group is a named set of nodes.
A Processing Profile names a node group and a set of periods, saying when processing may run and how many threads it may use in each period.
A processor filter can then be given a profile.

Together these let you keep heavy processing off the nodes that serve the user interface, hold it back until a quiet time of day, or give it more of the cluster at the weekend.

Node Groups

Node groups are managed from

Each group has:

Node Group Name - the name used to refer to the group from a processing profile.
Enabled - whether the group is in use. A disabled group does no processing at all, whatever its profiles say.
Node Inclusion Behaviour - whether the nodes you tick are included in the group or excluded from it.
Nodes - the nodes to tick.

The inclusion behaviour is worth understanding. Choosing exclude means the group is everything except the nodes you tick, so nodes added to the cluster later join the group automatically. Choosing include means only the ticked nodes are in the group, and a new node has to be added deliberately.

Processing Profiles

Processing profiles are managed from

Each profile has:

Profile Name - the name shown when choosing a profile on a processor filter.
Node Group - the group of nodes that may process for this profile.
Processing Schedules - one or more periods, described below.
Time Zone - the time zone the periods are interpreted in.

Setting the time zone explicitly matters for a cluster whose nodes are not all in the same place, and for periods that should follow local working hours across daylight saving changes.

Periods

Each period in the schedule has:

Active Days - the days of the week the period applies to.
Start Time and End Time - the time of day the period runs between. An end time that is not after the start time is treated as running into the following day, so a period can span midnight.
Limit Single Node Threads and Max Node Threads - whether to cap the threads used on each node, and the cap.
Limit Total Cluster Threads and Max Cluster Threads - whether to cap the threads used across the whole cluster, and the cap.

Leaving a limit unticked means that limit is not applied, i.e. the number of threads is unbounded for that period.

Periods are evaluated in order and the first one that matches the current day and time is used.

When No Period Matches

Warning

A filter that has been given a profile is processed only during the periods in that profile.

If no period matches the current day and time then no tasks are created for that filter. The same is true if the profile has no periods at all, if its node group is disabled, or if the node asking for work is not in the group.

This is the behaviour you want for confining processing to a window, but it does mean a profile with an incomplete schedule will quietly stop a filter from processing. A filter with no profile set is not restricted in this way.

Using a Profile on a Filter

A profile is chosen on the processor filter itself, using the Processing Profile field alongside the other filter settings such as Max Processing Tasks.

Task creation takes profiles into account, so tasks that no node is currently allowed to process are not added to the queue, and are released again if a profile stops allowing them. Each profile is given a share of the tasks that are created, so that one busy profile does not starve the others.

Stroom – node