Upgrade Notes

Required actions and information relating to upgrading to Stroom version 7.11.

Upgrade Path

You can upgrade to v7.11.x from any v7.x release that is older than the version being upgraded to.

If you want to upgrade to v7.11.x from v5.x or v6.x we recommend you do the following:

  1. Upgrade v5.x to the latest patch release of v6.0.
  2. Upgrade v6.x to the latest patch release of v7.0.
  3. Upgrade v7.x to the latest patch release of v7.11.

Java Version

Stroom v7.10 requires Java 25.

Ensure the Stroom and Stroom-Proxy hosts are running the latest patch release of Java v25.

Configuration File Changes

Common Configuration Changes

These changes are common to both Stroom and Stroom Proxy.

New receive Branch Properties

proxyConfig:
  receive:
    # The action to take if there is a problem with the data receipt rules, e.g.
    # Stroom Proxy has been unable to contact Stroom to fetch the rules.
    fallbackReceiveAction: "RECEIVE"
    # If defined then states the maximum size of a request (uncompressed for gzip requests).
    # Will return a 413 Content Too Long response code for any requests exceeding this
    # value. If undefined then there is no limit to the size of the request.
    # Defined as an IEC byte value, e.g. 10GiB, 10M, 23TB, 1024, etc.
    maxRequestSize: null
    # The data receipt checking mode, one of:
    # * FEED_STATUS - Use the legacy Feed Status Check method
    # * RECEIPT_POLICY - Use the new Data Receipt Rules
    # * RECEIVE_ALL - Receive ALL data with no checks
    # * DROP_ALL - Drop ALL data with no checks
    # * REJECT_ALL - Reject ALL data with no checks
    receiptCheckMode: "FEED_STATUS"

Removed Property proxyConfig.receive.receiptPolicyUuid

The property receiptPolicyUuid has been removed. This was a for an unused feature so its removal does not require any migration. If it is in your config file, simply remove it.

proxyConfig:
  receive:
    receiptPolicyUuid:

Stroom’s config.yml

New askStroomAi Branch

A new branch of configuration to configure Ask Stroom AI.

appCongfig:
  askStroomAi:
    chatMemory:
      # How long a chat memory entry should exist before being expired
      timeToLive:
        time: 1
        timeUnit: "HOURS"
      # Number of tokens to keep in each chat memory store
      tokenLimit: 30000
    # The DocRef of the OpenAIModel document to use for Ask Stroom AI. e.g.
    # modelRef:
    #  name: "My Model"
    #  type: "OpenAIModel"
    #  uuid: "8df699c0-2ce5-48bc-bb1a-e5d26bbd2175"
    modelRef:
    tableSummary:
      # Maximum number of tokens to pass the AI service at a time
      maximumBatchSize: 16384
      # Maximum number of table result rows to pass to the AI when making requests
      maximumTableInputRows: 100

New contentStore Branch

A new branch of configuration to configure one or more Content Stores.

appCongfig:
  contentStore:
    # A list of URLs to content store definition files
    urls:
    - "https://raw.githubusercontent.com/gchq/stroom-content/refs/heads/master/source/content-store.yml"

New credentials Branch

A new branch has been added for configuring the storage and caching of credentials for authenticating with other systems.

  credentials:
    # A standard configuration branch for a database connection.
    # Only set this if you want the credentials tables to be located on a different database
    # to the rest of Stroom.
    db:
      connection:
        #...
      connectionPool:
        # ...
    # The path to stored cached key stores.
    keyStoreCachePath: "${stroom.home}/keystores"

Removed Elastic Property

The property stroom.elastic.retention.scrollSize has been removed.

appCongfig:
  elastic:
    retention:           # <- REMOVED
      scrollSize: 10000  # <- REMOVED

db Branch added to gitRepo

A standard database configuration branch has been added to GitRepo. You should not need to set this unless you want the Git Repo table data to be stored on a different database.

appCongfig:
  gitRepo:
    db:
      connection:
        #...
      connectionPool:
        # ...

New receiptPolicy Branch

A new configuration branch for Data Receipt Rules.

  receiptPolicy:
    obfuscatedFields:
    - "AccountId"
    - "AccountName"
    # ...
    - "X-Forwarded-For"
    obfuscationHashAlgorithm: "SHA2_512"
    receiptRulesInitialFields:
      AccountId: "Text"
      Component: "Text"
    # ...
      user-agent: "Text"
      X-Forwarded-For: "Text"

Stroom-Proxy’s config.yml

Removed contentSync

The whole contentSync branch has been removed as it is no longer in use.

proxyConfig:
  contentSync:                  # <- REMOVED
    apiKey: null                # <- REMOVED
    contentSyncEnabled: false   # <- REMOVED
    syncFrequency: "PT1M"       # <- REMOVED
    upstreamUrl: null           # <- REMOVED

Added downstreamHost

A new downstreamHost branch has been added to configure the default downstream host details and authentication. In a typical deployment, Stroom Proxy will receive data and forward it on to a downstream Stroom or Stroom Proxy. There are a number of parts of Stroom Proxy that need to communicate with the downstream host, e.g. feed status checking, forwarding or rule fetching.

This allows for the host’s details and any authentication properties to be set in one place. Typically you will only need to set the hostname and apiKey properties.

proxyConfig:

  downstreamHost:
    # The API key to use for authentication (unless OpenID Connect is being used)
    apiKey: null
    # Only set this if you need to use a non-standard path
    apiKeyVerificationUrl: null
    enabled: true
    # The hostname of the downstream
    hostname: null
    # How long to cache verified keys for in memory
    maxCachedKeyAge: "PT10M"
    # How long keys will be persisted for on disk in case the downstream
    # can't be connected to
    maxPersistedKeyAge: "P30D"
    # The delay to use after there is an error connecting to the downstream
    noFetchIntervalAfterFailure: "PT30S"
    # Only requird if you need a common path prefix in front of the path
    # that Stroom / Stroom Proxy will use.
    pathPrefix: null
    # The hash algorithm used to hash persisted API keys.
    persistedKeysHashAlgorithm: "SHA2_512"
    # The port to connect to the downstream on
    # If not set, will default to 80/443 depending on scheme.
    port: null
    # The scheme to connect to the downstream on
    scheme: "https"

Remove various feedStatus properties

The following three properties have been removed from the feedStatus branch.

proxyConfig:
  feedStatus:
    apiKey: null             # <- REMOVED
    defaultStatus: "Receive" # <- REMOVED
    enabled: true            # <- REMOVED

The migration for these properties is as follows:

  • apiKey => proxyConfig.downstreamHost.apiKey
  • defaultStatus => proxyConfig.receive.fallbackReceiveAction
    • Receive => RECEIVE
    • Reject => REJECT
    • Drop => DROP
  • enabled => proxyConfig.receive.receiptCheckMode
    • true => FEED_STATUS
    • false => (RECEIPT_POLICY, RECEIVE_ALL, DROP_ALL, REJECT_ALL)

feedStatus.url

The url property for feed status checking no longer needs to be set unless you need to use a non-standard URL. The URL for feed status checking will now be derived from the properties in downstreamHost and the static path for the feed status resource. Therefore in most cases, simply remove this property.

Also, previously, the value for this property was just a path. Now, if you set it, it should be a full URL including host and path.

proxyConfig:
  feedStatus:
    url:

New forwardHttpDestinations property

A new property has been added to enable/disable the liveness checking for HTTP destinations. Liveness checking will periodically check that the destination is live and if not, disable forwarding until the liveness check determines it to be live again. Liveness checking is enabled by default.

proxyConfig:
  forwardHttpDestinations:
  - .....
    livenessCheckEnabled: true

New forwardFileDestinations property

A new property has been added to enable/disable the liveness checking for file destinations. Liveness checking will periodically check that the destination is live and if not, disable forwarding until the liveness check determines it to be live again. Liveness checking is enabled by default.

proxyConfig:
  forwardFileDestinations:
  - .....
    livenessCheckEnabled: true

Changes to forwardHttpDestinations[*].forwardUrl property

This property no longer needs to be set unless you need to forward to a location that is different to that defined by downstreamHost. The forward URL will now be derived from the properties in downstreamHost and the static path for the datafeed endpoint. Therefore in most cases, simply remove this property.

Also, if you need to set this property, the value of the forwardUrl property has changed from being just a path to being a full URL including host and path.

proxyConfig:
  forwardHttpDestinations:
  - .....
    forwardUrl:

Added receiptPolicy Branch

This configuration branch has been added to configure the Data Receipt Rules.

proxyConfig:
  receiptPolicy:
    # Stroom Proxy will use downstreamHost to derive the URL to connect to.
    # Only set this if you need to use a non-standard URL.
    receiveDataRulesUrl: null
    # The frequency that Stroom Proxy will connect to the downstream host to obtain updated
    # receipt policy rules.
    syncFrequency: "PT1M"

Database Migrations

When Stroom boots for the first time with a new version it will run any required database migrations to bring the database schema up to the correct version.

On boot, Stroom will ensure that the migrations are only run by a single node in the cluster. This will be the node that reaches that point in the boot process first. All other nodes will wait until that is complete before proceeding with the boot process.

It is recommended however to use a single node to execute the migration. To avoid Stroom starting up and beginning processing you can use the migrate command to just migrate the database and not fully boot Stroom. See migrate command for more details.

Pre-Migration Scripts

A previous database migration in v7.6 may have resulted in annotation_entry database records with no value in the entry_user_uuid column. This is likely caused by annotations being linked to users that were no longer in Stroom when the 7.6 migration ran. If that is the case then the v7.11 migration V07_11_00_001__annotation3.sql will fail.

To establish if you need you have any records in this state, run the following script against the Stroom database. This script can also be downloaded from v07_11_db_pre_migration_checks.sql on GitHub.

-- A set of SQL queries to run before migrating from v6 to v7
-- See https://gchq.github.io/stroom-docs/releases/v07.11/upgrade-notes/
-- 
-- Run with the mysql --table arg to get formatted output
-- e.g.
-- mysql --force --table -h"localhost" -P"3306" -u"stroomuser" -p"stroompassword1" stroom < v07_11_db_pre_migration_checks.sql > v07_11_db_pre_migration_checks.out 2>&1
-- docker exec -i stroom-all-dbs mysql --force --table -h"localhost" -P"3307" -u"stroomuser" -p"stroompassword1" stroom < v07_11_db_pre_migration_checks.sql > v07_11_db_pre_migration_checks.out 2>&1

\! echo 'Find annotation entries with no assigned user. No action required if this returns nothing.';

SELECT a.id, a.title, ae.id AS entry_id
FROM annotation a
INNER JOIN annotation_entry ae ON a.id = ae.fk_annotation_id
WHERE ae.entry_user_uuid IS NULL
OR ae.entry_user_uuid = '';

\! echo 'Listing all enabled Stroom users for reference';

SELECT uuid, name, display_name, full_name
FROM stroom_user su
WHERE is_group = false
AND enabled = true
ORDER by name;

\! echo 'Finished';

If the first SELECT statement returns any rows you will need to take remedial action.

In order to fix the missing data, you need the UUID UUID A Universally Unique Identifier for uniquely identifying something. UUIDs are used as the identifier in Doc Refs. An example of a UUID is 4ffeb895-53c9-40d6-bf33-3ef025401ad3.Click to see more details... of a user that can be associated with the annotation entries. You will need to establish the UUID UUID A Universally Unique Identifier for uniquely identifying something. UUIDs are used as the identifier in Doc Refs. An example of a UUID is 4ffeb895-53c9-40d6-bf33-3ef025401ad3.Click to see more details... of a suitable user, for example a user that is an administrator. The above script will also list all enabled users in your Stroom instance.

Select the uuid of an appropriate user and run the following SQL SQL Structured Query Language (SQL). The language used in the MySQL database to query and manipulate the data.Click to see more details... statement, replacing <insert user UUID here> with the chosen UUID value.

UPDATE annotation_entry
SET entry_user_uuid = '<insert user UUID here>'
WHERE entry_user_uuid IS NULL
OR entry_user_uuid = '';

Once complete, re-run the v07_11_db_pre_migration_checks.sql script to verify that the column is now fully populated. Then proceed with the 7.11 migration.

Migration Scripts

For information purposes only, the following are the database migrations that will be run when upgrading to 7.11.0 from the previous minor version.

Note, the legacy module will run first (if present) then the other module will run in no particular order.

Module stroom-annotation

Script V07_11_00_001__annotation3.sql

Path: stroom-annotation/stroom-annotation-impl-db/src/main/resources/stroom/annotation/impl/db/migration/V07_11_00_001__annotation3.sql

-- ------------------------------------------------------------------------
-- Copyright 2023 Crown Copyright
--
-- Licensed under the Apache License, Version 2.0 (the "License");
-- you may not use this file except in compliance with the License.
-- You may obtain a copy of the License at
--
--     http://www.apache.org/licenses/LICENSE-2.0
--
-- Unless required by applicable law or agreed to in writing, software
-- distributed under the License is distributed on an "AS IS" BASIS,
-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-- See the License for the specific language governing permissions and
-- limitations under the License.
-- ------------------------------------------------------------------------

-- Stop NOTE level warnings about objects (not)? existing
SET @OLD_SQL_NOTES=@@SQL_NOTES, SQL_NOTES=0;

DROP PROCEDURE IF EXISTS V07_11_00_001_annotation;

DELIMITER $$

CREATE PROCEDURE V07_11_00_001_annotation ()
BEGIN
    DECLARE object_count integer;

    --
    -- Add entry parent id
    --
    SELECT COUNT(1)
    INTO object_count
    FROM information_schema.columns
    WHERE table_schema = database()
    AND table_name = 'annotation_entry'
    AND column_name = 'parent_id';

    IF object_count = 0 THEN
        ALTER TABLE `annotation_entry` ADD COLUMN `parent_id` bigint DEFAULT NULL;
    END IF;

    --
    -- Add entry update time
    --
    SELECT COUNT(1)
    INTO object_count
    FROM information_schema.columns
    WHERE table_schema = database()
    AND table_name = 'annotation_entry'
    AND column_name = 'update_time_ms';

    IF object_count = 0 THEN
        ALTER TABLE `annotation_entry` ADD COLUMN `update_time_ms` bigint(20) NOT NULL;

        -- Copy all entry times to update times.
        SET @sql_str = CONCAT(
            'UPDATE annotation_entry a ',
            'SET a.update_time_ms = a.entry_time_ms');
        PREPARE stmt FROM @sql_str;
        EXECUTE stmt;

    END IF;

    --
    -- Add entry update user
    --
    SELECT COUNT(1)
    INTO object_count
    FROM information_schema.columns
    WHERE table_schema = database()
    AND table_name = 'annotation_entry'
    AND column_name = 'update_user_uuid';

    IF object_count = 0 THEN
        ALTER TABLE `annotation_entry` ADD COLUMN `update_user_uuid` varchar(255) NOT NULL;

        -- Copy all entry users to update users.
        SET @sql_str = CONCAT(
            'UPDATE annotation_entry a ',
            'SET a.update_user_uuid = a.entry_user_uuid');
        PREPARE stmt FROM @sql_str;
        EXECUTE stmt;

    END IF;

END $$

DELIMITER ;

CALL V07_11_00_001_annotation;

DROP PROCEDURE IF EXISTS V07_11_00_001_annotation;

SET SQL_NOTES=@OLD_SQL_NOTES;

-- vim: set shiftwidth=4 tabstop=4 expandtab:

Module stroom-credentials

Script V07_11_00_001__credential.sql

Path: stroom-credentials/stroom-credentials-impl-db/src/main/resources/stroom/credentials/impl/db/migration/V07_11_00_001__credential.sql

-- ------------------------------------------------------------------------
-- Copyright 2025 Crown Copyright
--
-- Licensed under the Apache License, Version 2.0 (the "License");
-- you may not use this file except in compliance with the License.
-- You may obtain a copy of the License at
--
--     http://www.apache.org/licenses/LICENSE-2.0
--
-- Unless required by applicable law or agreed to in writing, software
-- distributed under the License is distributed on an "AS IS" BASIS,
-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-- See the License for the specific language governing permissions and
-- limitations under the License.
-- ------------------------------------------------------------------------

-- Stop NOTE level warnings about objects (not)? existing
SET @OLD_SQL_NOTES=@@SQL_NOTES, SQL_NOTES=0;

--
-- Create the credential tables
--
CREATE TABLE IF NOT EXISTS credential (
    uuid                  varchar(255) NOT NULL,
    create_time_ms        bigint(20) NOT NULL,
    create_user           varchar(255) NOT NULL,
    update_time_ms        bigint(20) NOT NULL,
    update_user           varchar(255) NOT NULL,
    name                  varchar(255) NOT NULL,
    crendential_type      varchar(255) NOT NULL,
    key_store_type        varchar(255) DEFAULT NULL,
    expiry_time_ms        bigint DEFAULT NULL,
    secret_json           json NOT NULL,
    key_store             longblob DEFAULT NULL,
    PRIMARY KEY  (uuid),
    UNIQUE KEY name (name)
) ENGINE=InnoDB DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_0900_ai_ci;

SET SQL_NOTES=@OLD_SQL_NOTES;

-- vim: set shiftwidth=4 tabstop=4 expandtab:

Module stroom-index

Script V07_11_00_001__index_field.sql

Path: stroom-index/stroom-index-impl-db/src/main/resources/stroom/index/impl/db/migration/V07_11_00_001__index_field.sql

-- Stop NOTE level warnings about objects (not)? existing
SET @OLD_SQL_NOTES=@@SQL_NOTES, SQL_NOTES=0;

ALTER TABLE index_field ADD COLUMN `dense_vector` json DEFAULT NULL;

SET SQL_NOTES=@OLD_SQL_NOTES;

Module stroom-processor

Script V07_11_00_001__processor_filter.sql

Path: stroom-processor/stroom-processor-impl-db/src/main/resources/stroom/processor/impl/db/migration/V07_11_00_001__processor_filter.sql

-- ------------------------------------------------------------------------
-- Copyright 2020 Crown Copyright
--
-- Licensed under the Apache License, Version 2.0 (the "License");
-- you may not use this file except in compliance with the License.
-- You may obtain a copy of the License at
--
--     http://www.apache.org/licenses/LICENSE-2.0
--
-- Unless required by applicable law or agreed to in writing, software
-- distributed under the License is distributed on an "AS IS" BASIS,
-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-- See the License for the specific language governing permissions and
-- limitations under the License.
-- ------------------------------------------------------------------------

-- Stop NOTE level warnings about objects (not)? existing
SET @OLD_SQL_NOTES=@@SQL_NOTES, SQL_NOTES=0;

-- --------------------------------------------------

DELIMITER $$

-- --------------------------------------------------

DROP PROCEDURE IF EXISTS processor_run_sql_v1 $$

-- DO NOT change this without reading the header!
CREATE PROCEDURE processor_run_sql_v1 (
    p_sql_stmt varchar(1000)
)
BEGIN

    SET @sqlstmt = p_sql_stmt;

    SELECT CONCAT('Running sql: ', @sqlstmt);

    PREPARE stmt FROM @sqlstmt;
    EXECUTE stmt;
    DEALLOCATE PREPARE stmt;
END $$

-- --------------------------------------------------

DROP PROCEDURE IF EXISTS processor_add_column_v1$$

-- DO NOT change this without reading the header!
CREATE PROCEDURE processor_add_column_v1 (
    p_table_name varchar(64),
    p_column_name varchar(64),
    p_column_type_info varchar(64) -- e.g. 'varchar(255) default NULL'
)
BEGIN
    DECLARE object_count integer;

    SELECT COUNT(1)
    INTO object_count
    FROM information_schema.columns
    WHERE table_schema = database()
    AND table_name = p_table_name
    AND column_name = p_column_name;

    IF object_count = 0 THEN
        CALL processor_run_sql_v1(CONCAT(
            'alter table ', database(), '.', p_table_name,
            ' add column ', p_column_name, ' ', p_column_type_info));
    ELSE
        SELECT CONCAT(
            'Column ',
            p_column_name,
            ' already exists on table ',
            database(),
            '.',
            p_table_name);
    END IF;
END $$

-- idempotent
CALL processor_add_column_v1(
        'processor_filter',
        'export',
        'TINYINT(1) NOT NULL DEFAULT 0')$$

-- vim: set shiftwidth=4 tabstop=4 expandtab:
Last modified April 24, 2026: Fill in 7.12 upgrade notes (597c57e)