Version 7.8

• 1: New Features
• 2: Preview Features (experimental)
• 3: Breaking Changes
• 4: Upgrade Notes
• 5: Change Log

1 - New Features

New Stroom-Proxy implementation

Stroom-Proxy has been re-written to fundamentally change the way it works internally. This is to improve performance over previous Stroom-Proxy implementations and to make it more robust.

Reporting

Apache HTTP Client

Previous versions of Stroom and Stroom-Proxy used a variety of HTTP clients for making HTTP requests, e.g. from the HTTPAppender . These have been standardised to all use the Apache HTTP client which means a consistent set of configuration can be used.

Receipt ID

A new meta attribute (ReceiptId) has been added to Stroom and Stroom-Proxy to provide a unique identifier for each request received.

The format of this attribute has been made to make it more useful to administrators, while still being unique across the environment that the Stroom and Stroom-Proxy instances are deployed in.

The format is as follows:

<timestamp>_<seq no>_<(P|S)>_<proxyId or stroom nodeName>

• <timestamp> - The receipt timestamp in milliseconds since the Unix Epoch , zero padded.
• <seq no> - This is zero padded four digit sequential number (starting at 0000) that is used to distinguish between multiple receipt events happening during the same millisecond.
• <P|S> - Indicates whether the item was received by Stroom (S) or Stroom-Proxy (P).
• <proxyId or stroom nodeName> - For Stroom-Proxy this will be the proxyId that is either set in configuration to uniquely identify a proxy instance or is one of the FQDN/IP. For Stroom this is the node name of the Stroom instance.

An example Receipt ID is 0000001738332835967_0000_P_node1

The new format is useful for tracing the flow of data through a chain of proxies as it will be included in receive and send logs as well as being written to the meta attributes.

In addition to ReceiptId we also have ReceiptIdPath which will have a ReceiptId appended to it on each receipt (comma delimited).

To ensure uniqueness of these IDs across the estate, ProxyIDs should be unique within the environment that data will flow. The same is true for Stroom node names.

2 - Preview Features (experimental)

There are no preview features in v7.8.

3 - Breaking Changes

Stroom

Service Discovery

See Configuration File Changes (Service Discovery).

Stroom-Proxy

The new implementation of Stroom-Proxy is not compatible with previous versions. It uses a different structure for its local repository. Therefor you CANNOT deploy Stroom-Proxy v7.8 on top of a previous Stroom-Proxy version as it will be unable to process existing stored data that has not yet been forwarded.

You have two options:

1. Run the old and new in parallel until the old one has processed all its data, then shutdown and uninstall the old one.

2. Prevent data going to the old Stroom-Proxy and allow it to process all its data. Shut it down and remove the installation. Install the new v7.8 Stroom-Proxy. Allow data to flow in again.

4 - Upgrade Notes

Java Version

Stroom v7.8 requires Java 21. This is the same java version as Stroom v7.7. Ensure the Stroom and Stroom-Proxy hosts are running the latest patch release of Java v21.

Configuration File Changes

Stroom’s config.yml

Service Discovery

If your config.yml file contains the serviceDiscovery branch, you need to remove it as the associated service discovery code has been removed from Stroom. If you do not remove it, Stroom will not boot.

appConfig:
  serviceDiscovery:  # Remove this branch

Plan B State Store

The following block of config has been added for the experimental Plan B State Store.

appConfig:
  planb:
    minTimeToKeepEnvOpen: "PT1M"
    minTimeToKeepSnapshots: "PT10M"
    nodeList: []
    path: "${stroom.home}/planb"
    snapshotRetryFetchInterval: "PT1M"
    stateDocCache:
      expireAfterAccess: null
      expireAfterWrite: "PT10M"
      maximumSize: 100
      refreshAfterWrite: null

Reporting

The following block of config has been added for the new reporting feature.

appConfig:
  ui:
    reportUiDefaultConfig:
      defaultBodyTemplate: "<!DOCTYPE html>\n<html lang=\"en\">\n<meta charset=\"\
        UTF-8\" />\n<title>Report '{{ reportName | escape }}'</title>\n<body>\n <p><em>Report:\
        \ {{ reportName | escape }}</em>  executed for {{ effectiveExecutionTime |\
        \ escape }} on {{ executionTime | escape }}</p>\n <p><em>Description:</em>\
        \  {{ description | escape }}</p>\n</body>\n"
      defaultSubjectTemplate: "Report '{{ reportName | escape }}'"

OpenID HTTP Client

Added the property httpClient for configuring the HTTP client used to fetch the IDP's configuration.

  security:
    authentication:
      openId:
        httpClient: null

Stroom-Proxy’s config.yml

The configuration for proxy has changed significantly since the last stable release of Stroom-Proxy (7.0.x) so it is advised to look at the default Stroom-Proxy configuration that comes with the distribution.

Database Migrations

When Stroom boots for the first time with a new version it will run any required database migrations to bring the database schema up to the correct version.

On boot, Stroom will ensure that the migrations are only run by a single node in the cluster. This will be the node that reaches that point in the boot process first. All other nodes will wait until that is complete before proceeding with the boot process.

It is recommended however to use a single node to execute the migration. To avoid Stroom starting up and beginning processing you can use the migrage command to just migrate the database and not fully boot Stroom. See migrage command for more details.

Migration Scripts

There are no database migrations in v7.8.

5 - Change Log

• Issue #4682 : Improve Plan B filter error handling.

• Change receiptId format to <epoch ms>_<seq no>_<(P|S)>_<proxyId or stroom nodeName>. P|S represents stroom or Proxy.

• Change stroom to also set the receiptId meta attribute on receipt or upload.

• Change proxy logging to still log datafeed events even if the metaKeys config prop is empty.

• Issue #4695 : Change proxy to re-create the proxy ID in proxy-id.txt if the value in there does not match the required pattern.

• Fix the sleep time in UniqueIdGenerator (from 50ms to 0.1ms).

• Fix tests that were breaking the build.

• Change the format of proxy receiptIds. They now look like 0000001736533752496_0000_TestProxy with the first part being the epoch millis for when it was generated and the last part is the proxy ID.

• Change default identityProviderType to INTERNAL_IDP in stroom prod config.

• Issue #4661 : Use Apache HttpClient.

• Issue #4378 : Add reporting.

• Issue #2201 : New proxy implementation.