Class GafferAdder
- java.lang.Object
-
- uk.gov.gchq.gaffer.flink.operation.handler.GafferAdder
-
- All Implemented Interfaces:
Serializable
public class GafferAdder extends Object implements Serializable
Helper class to add
Element
s to a Gaffer store.The Flink sink is given a single element at a time. Adding a single element at a time to Gaffer would be really inefficient so we add these individual elements to a blocking queue that we simultaneously add elements to whilst the Gaffer Store is consuming them.
The queue is a
BlockingQueue
with a maximum size to prevent the queue from getting too large and running out of memory. If the maximum size is reached Flink will be blocked from adding elements to the queue. The maximum size of the queue can be configured using the operation option: gaffer.flink.operation.handler.max-queue-size. By default the maximum size is 1,000,000. The blocking queue is only blocked on addition, it does not cause the Gaffer Store to block if the queue is empty. In the case where a Kafka queue has just a single Gaffer element the Store can immediately add this rather than blocking and waiting for a batch to fill up. A side affect of this is that AddElements operation may complete if no elements are added to the queue for some time. In this situation we just restart the AddElements operation next time an element is received.- See Also:
- Serialized Form
-
-
Field Summary
Fields Modifier and Type Field Description static int
MAX_QUEUE_SIZE_DEFAULT
-
Constructor Summary
Constructors Constructor Description GafferAdder(OP operation, Store store)
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description void
add(Element element)
void
initialise()
-
-
-
Field Detail
-
MAX_QUEUE_SIZE_DEFAULT
public static final int MAX_QUEUE_SIZE_DEFAULT
- See Also:
- Constant Field Values
-
-
Constructor Detail
-
GafferAdder
public GafferAdder(OP operation, Store store)
-
-
Method Detail
-
initialise
public void initialise()
-
add
public void add(Element element)
-
-