Package uk.gov.gchq.gaffer.sparkaccumulo.operation.handler.scalardd

Class GetRDDOfAllElementsHandler

java.lang.Object

uk.gov.gchq.gaffer.sparkaccumulo.operation.handler.AbstractGetRDDHandler<GetRDDOfAllElements,org.apache.spark.rdd.RDD<Element>>

uk.gov.gchq.gaffer.sparkaccumulo.operation.handler.scalardd.GetRDDOfAllElementsHandler

All Implemented Interfaces:

OperationHandler<GetRDDOfAllElements>, OutputOperationHandler<GetRDDOfAllElements,org.apache.spark.rdd.RDD<Element>>

public class GetRDDOfAllElementsHandler
extends AbstractGetRDDHandler<GetRDDOfAllElements,org.apache.spark.rdd.RDD<Element>>

A handler for the GetRDDOfAllElements operation.

If the gaffer.accumulo.spark.directrdd.use_rfile_reader option is set to true then the RDD will be produced by directly reading the RFiles in the Accumulo table, rather than using ElementInputFormat to get data via the tablet servers. In order to read the RFiles directly, the user must have read access to the files. Also note that any data that has not been minor compacted will not be read. Reading the Rfiles directly can increase the performance.

If the gaffer.accumulo.spark.directrdd.use_rfile_reader option is not set then the standard approach of obtaining data via the tablet servers is used.

When reading data via the tablet servers, read performance may be improved by setting the gaffer.accumulo.spark.rdd.use_batch_scanner option to true.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	GetRDDOfAllElementsHandler.EntryIteratorToElementIterator
static class	GetRDDOfAllElementsHandler.EntryToElement
static class	GetRDDOfAllElementsHandler.FilterOutNull

Nested classes/interfaces inherited from class uk.gov.gchq.gaffer.sparkaccumulo.operation.handler.AbstractGetRDDHandler

AbstractGetRDDHandler.FirstElement

Field Summary

Fields inherited from class uk.gov.gchq.gaffer.sparkaccumulo.operation.handler.AbstractGetRDDHandler

HADOOP_CONFIGURATION_KEY, USE_BATCH_SCANNER_RDD, USE_RFILE_READER_RDD, VIEW

Constructor Summary

Constructors

Constructor	Description
GetRDDOfAllElementsHandler()

Method Summary

Modifier and Type	Method	Description
org.apache.spark.rdd.RDD<Element>	doOperation(GetRDDOfAllElements operation, Context context, Store store)	Execute the given Output operation.

Methods inherited from class uk.gov.gchq.gaffer.sparkaccumulo.operation.handler.AbstractGetRDDHandler

addIterators, addRanges, addRangesFromPairs, convertConfigurationToString, convertStringToConfiguration

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

GetRDDOfAllElementsHandler

public GetRDDOfAllElementsHandler()

Method Detail

doOperation

public org.apache.spark.rdd.RDD<Element> doOperation(GetRDDOfAllElements operation,
                                                     Context context,
                                                     Store store)
                                              throws OperationException

Description copied from interface: OutputOperationHandler

Execute the given Output operation.

Parameters:

operation - the Output operation to be executed

context - the operation chain context, containing the user who executed the operation

store - the Store the operation should be run on

Returns:

the output for the operation.

Throws:

OperationException - thrown if the operation fails