Package uk.gov.gchq.gaffer.accumulostore.utils

Class IngestUtils

java.lang.Object

uk.gov.gchq.gaffer.accumulostore.utils.IngestUtils

public final class IngestUtils
extends Object

Utility methods for adding data to Accumulo.

Method Summary

Modifier and Type	Method	Description
static int	createSplitsFile(org.apache.accumulo.core.client.Connector conn, String table, org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path splitsFile)
static int	createSplitsFile(org.apache.accumulo.core.client.Connector conn, String table, org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path splitsFile, int maxSplits)	Get the existing splits from a table in Accumulo and write a splits file.
static int	getNumSplits(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path splitsFile)	Read a splits file and get the number of split points within
static void	setDirectoryPermsForAccumulo(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path dirPath)	Modify the permissions on a directory and its contents to allow Accumulo access.

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

createSplitsFile

public static int createSplitsFile(org.apache.accumulo.core.client.Connector conn,
                                   String table,
                                   org.apache.hadoop.fs.FileSystem fs,
                                   org.apache.hadoop.fs.Path splitsFile,
                                   int maxSplits)
                            throws IOException

Get the existing splits from a table in Accumulo and write a splits file. The number of splits is returned.

Parameters:

conn - - An existing connection to an Accumulo instance

table - - The table name

fs - - The FileSystem in which to create the splits file

splitsFile - - A path for the output splits file

maxSplits - - The maximum number of splits

Returns:

The number of splits in the table

Throws:

IOException - for any IO issues reading from the file system. Other accumulo exceptions are caught and wrapped in an IOException.

createSplitsFile

public static int createSplitsFile(org.apache.accumulo.core.client.Connector conn,
                                   String table,
                                   org.apache.hadoop.fs.FileSystem fs,
                                   org.apache.hadoop.fs.Path splitsFile)
                            throws IOException

Throws:

IOException

getNumSplits

public static int getNumSplits(org.apache.hadoop.fs.FileSystem fs,
                               org.apache.hadoop.fs.Path splitsFile)
                        throws IOException

Read a splits file and get the number of split points within

Parameters:

fs - - The FileSystem in which to create the splits file

splitsFile - - A path for the output splits file

Returns:

An integer representing the number of entries in the file.

Throws:

IOException - for any IO issues reading from the file system.

setDirectoryPermsForAccumulo

public static void setDirectoryPermsForAccumulo(org.apache.hadoop.fs.FileSystem fs,
                                                org.apache.hadoop.fs.Path dirPath)
                                         throws IOException

Modify the permissions on a directory and its contents to allow Accumulo access.

Parameters:

fs - - The FileSystem in which to create the splits file

dirPath - - The path to the directory

Throws:

IOException - for any IO issues interacting with the file system.