Package uk.gov.gchq.gaffer.accumulostore.operation.hdfs.handler.job.partitioner

Class GafferKeyRangePartitioner

java.lang.Object

org.apache.hadoop.mapreduce.Partitioner<org.apache.accumulo.core.data.Key,org.apache.hadoop.io.Writable>

uk.gov.gchq.gaffer.accumulostore.operation.hdfs.handler.job.partitioner.GafferKeyRangePartitioner

All Implemented Interfaces:

org.apache.hadoop.conf.Configurable

public class GafferKeyRangePartitioner
extends org.apache.hadoop.mapreduce.Partitioner<org.apache.accumulo.core.data.Key,org.apache.hadoop.io.Writable>
implements org.apache.hadoop.conf.Configurable

Copy of KeyRangePartitioner and swaps the RangePartitioner for the GafferRangePartitioner to fix a bug with opening the split points file.

Constructor Summary

Constructors

Constructor	Description
GafferKeyRangePartitioner()

Method Summary

Modifier and Type	Method	Description
org.apache.hadoop.conf.Configuration	getConf()
int	getPartition(org.apache.accumulo.core.data.Key key, org.apache.hadoop.io.Writable value, int numPartitions)
void	setConf(org.apache.hadoop.conf.Configuration conf)
static void	setNumSubBins(org.apache.hadoop.mapreduce.Job job, int num)	Sets the number of random sub-bins per range
static void	setSplitFile(org.apache.hadoop.mapreduce.Job job, String file)	Sets the hdfs file name to use, containing a newline separated list of Base64 encoded split points that represent ranges for partitioning

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

GafferKeyRangePartitioner

public GafferKeyRangePartitioner()

Method Detail

getPartition

public int getPartition(org.apache.accumulo.core.data.Key key,
                        org.apache.hadoop.io.Writable value,
                        int numPartitions)

Specified by:

getPartition in class org.apache.hadoop.mapreduce.Partitioner<org.apache.accumulo.core.data.Key,org.apache.hadoop.io.Writable>

getConf

public org.apache.hadoop.conf.Configuration getConf()

Specified by:

getConf in interface org.apache.hadoop.conf.Configurable

setConf

public void setConf(org.apache.hadoop.conf.Configuration conf)

Specified by:

setConf in interface org.apache.hadoop.conf.Configurable

setSplitFile

public static void setSplitFile(org.apache.hadoop.mapreduce.Job job,
                                String file)

Sets the hdfs file name to use, containing a newline separated list of Base64 encoded split points that represent ranges for partitioning

Parameters:

job - the job

file - the splits file

setNumSubBins

public static void setNumSubBins(org.apache.hadoop.mapreduce.Job job,
                                 int num)

Sets the number of random sub-bins per range

Parameters:

job - the job

num - the number of sub bins