Skip to content

Gaffer 2.0.0 Changelist

Below is a summary of changes that have been made in Gaffer version 2.0.0.

Each breaking change has been put into one of 3 tiers:

  • user: end users might have to change their Gaffer queries
  • admin: Gaffer admins that manage graphs might have to change their config or deployment environment
  • developer: developers of Gaffer or Gaffer extensions might need to make code changes

Changes that are relevant to users are also relevant to admins and developers. Changes that are relevant to admins are also relevant to developers.

Accumulo 2 Support

The Accumulo store now supports Accumulo 2 and Hadoop 3 by default, with support for Accumulo 1 and Hadoop 2 retained. See the Accumulo Migration page for more information about this change.

Federated Store Improvements

The Federated Operation was added to greatly improve flexibility of using a Federated Store.

Breaking change | user | alpha-0.4

To migrate, please see the Federated Store Changes page.

Cache Improvements and fixes

All "caches" within Gaffer received a lot of bug fixes which should make them significantly more stable and consistent over time. This should improve usability of FederatedStores, NamedOperations and NamedViews.

Breaking change | admin | alpha-0.6

The cache will need to be reloaded, as the new internal cache interface has changed. To do this, export all of the contents of your cache, upgrade, then re-add everything manually.

Removal of Deprecated code

All of Gaffer 1's deprecated code has been removed.

Breaking change | user | alpha-0.1

To migrate, please see the deprecations page.

Dependency Upgrades

Dependencies have been updated, where possible to the latest version, removing vulnerabilities.

Breaking change | developer | alpha-0.2

You will need to migrate your dependencies to be compatible with Gaffer 2's new dependency versions. Please see the dependencies page for full details.

Federated and Proxy store fixes

A lot of bugs have been fixed that should facilitate FederatedStores with ProxyStores in them.

Breaking change | developer | alpha-0.5

The unique store trait DYNAMIC_SCHEMA has been removed from Gaffer. Simply removing it from custom FederatedStore implementations should be an adequate fix.

Removal of CloseableIterable and CloseableIterator

The CloseableIterable class has been removed so Operations like GetAllElements now return an Iterable instead, but the result still implements Closeable.

Breaking change | developer | alpha-0.2

Everywhere CloseableIterable was used in client code should be replaced with an Iterable: 

final CloseableIterable<? extends Element> results = graph.execute(new GetAllElements(), USER);

final Iterable<? extends Element> results = graph.execute(new GetAllElements(), USER);

Note that the CloseableIterator class has similarly been removed and any implementations will require updating.

Migration of Iterable Classes

Some Gaffer-specific Iterables and Iterators used for retrieving and manipulating data from stores have migrated from uk.gov.gchq.gaffer.commonutil.iterable to uk.gov.gchq.koryphe.iterable.

ChainedIterable, ChainedIterator, FilteredIterable, FilteredIterator, LimitedIterable (previously LimitedCloseableIterable) and LimitedIterator have been migrated into the Koryphe repo. Additionally, the MappedIterable and MappedIterator classes have been added which allow you to apply a list of functions to an iterable/iterator.

Many of the other classes remain in uk.gov.gchq.gaffer.commonutil.iterable.

Both the WrappedCloseableIterable and WrappedCloseableIterator classes have been removed.

Removal of HBase and Parquet stores

The HBase and Parquet stores have been removed from Gaffer in version 2. We made posts for both the HBase and Parquet stores to understand the levels of usage. It was then decided to remove both stores as this would make introducing various improvements easier in the long term. HBase and Parquet remain available in Gaffer version 1. In the future, they could be reimplemented for Gaffer 2, though we do not plan to currently.

Breaking change | admin | alpha-0.1

We would recommend instead using an Accumulo Store. If you would like these store implementations in Gaffer 2, or any other potential store for that matter, please make an issue on GitHub.

Gaffer now builds with Java 8 and Java 11

There is now a maven profile that will swap dependency versions so you can build Gaffer with Java 11. The code has also been updated to build with both Java versions.

Accumulo Kerberos Authentication Support

The Accumulo store now supports authenticating to Accumulo and HDFS using Kerberos, in addition to username/password. For more information, see the Kerberos support page.

CSV Import and Export

Basic support for importing and exporting CSVs has been added.

All operations can now be used within NamedOperations

Previously, GetElementsBetweenSets could not be used within a NamedOperation as it used inputB. GetElementsBetweenSets and inputB have both been deprecated and instead you should use GetElementsBetweenSetsPairs.

Example

Old operation now deprecated: 

{
    "class": "GetElementsBetweenSets",
    "input": [
        {
            "class": "EntitySeed",
            "vertex": "firstInput"
        }
    ],
    "inputB": [
        {
            "class": "EntitySeed",
            "vertex": "secondInput"
        }
    ]
}
New operation that will work within NamedOperations: 
{
    "class": "GetElementsBetweenSetsPairs",
    "input": {
        "class": "Pair",
        "first": {
            "ArrayList" : [
                {
                    "class": "EntitySeed",
                    "vertex": "firstInput"
                }
            ]
        },
        "second": {
            "ArrayList" : [
                {
                    "class": "EntitySeed",
                    "vertex": "secondInput"
                }
            ]
        }
    }
}

Ability to set OperationDeclarations during AddGraph

This will mean subgraphs added to FederatedStores can have additional operation handlers set when they are added. You can directly provide the OperationsDeclarations json to the store properties with gaffer.store.operation.declarations.json.

Example 
{
    "class": "AddGraph",
    "graphId": "myGraph",
    "schema": {}, // (1)!
    "storeProperties": {
        "gaffer.store.class": "MapStore",
        "gaffer.store.operation.declarations.json": {
            "operations": [
                {
                    "operation": "ImportFromLocalFile", // (2)!
                    "handler": {
                        "class": "ImportFromLocalFileHandler"
                    }
                }
            ]
        }
    }
}
  1. Schema left empty for brevity
  2. This example operation enables file import. Read more in the CSV docs.