Package uk.gov.gchq.gaffer.operation.impl.add

Class AddElementsFromKafka

java.lang.Object

uk.gov.gchq.gaffer.operation.impl.add.AddElementsFromKafka

All Implemented Interfaces:

Closeable, AutoCloseable, Operation, Validatable

public class AddElementsFromKafka
extends Object
implements Operation, Validatable

An AddElementsFromKafka operation consumes records of a Kafka topic, converts each record into a Gaffer Element using the provided ElementGenerator then adds these elements to the Graph. This operation is a blocking operation and will never stop. You will need to terminate the job when you want to stop consuming data.

See Also:

AddElementsFromKafka.Builder

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	AddElementsFromKafka.Builder

Nested classes/interfaces inherited from interface uk.gov.gchq.gaffer.operation.Operation

Operation.BaseBuilder<OP extends Operation,B extends Operation.BaseBuilder<OP,?>>

Field Summary

Fields

Modifier and Type	Field	Description
static Class<String>	DEFAULT_CONSUME_AS

Constructor Summary

Constructors

Constructor	Description
AddElementsFromKafka()

Method Summary

Modifier and Type	Method	Description
String[]	getBootstrapServers()
Class<?>	getConsumeAs()
Class<? extends Function<Iterable<?>,Iterable<? extends Element>>>	getElementGenerator()
String	getGroupId()
Map<String,String>	getOptions()
Integer	getParallelism()
String	getTopic()
boolean	isSkipInvalidElements()
boolean	isValidate()
void	setBootstrapServers(String... bootstrapServers)
void	setConsumeAs(Class<?> consumeAs)
void	setElementGenerator(Class<? extends Function<Iterable<?>,Iterable<? extends Element>>> elementGenerator)
void	setGroupId(String groupId)
void	setOptions(Map<String,String> options)
void	setParallelism(Integer parallelism)
void	setSkipInvalidElements(boolean skipInvalidElements)
void	setTopic(String topic)
void	setValidate(boolean validate)
AddElementsFromKafka	shallowClone()	Operation implementations should ensure a ShallowClone method is implemented.
uk.gov.gchq.koryphe.ValidationResult	validate()	Validates an operation.

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface uk.gov.gchq.gaffer.operation.Operation

_getNullOrOptions, addOption, close, containsOption, getOption, getOption, validateRequiredFieldPresent

Field Detail

DEFAULT_CONSUME_AS

public static final Class<String> DEFAULT_CONSUME_AS

Constructor Detail

AddElementsFromKafka

public AddElementsFromKafka()

Method Detail

getTopic

public String getTopic()

setTopic

public void setTopic(String topic)

getGroupId

public String getGroupId()

setGroupId

public void setGroupId(String groupId)

setParallelism

public void setParallelism(Integer parallelism)

getParallelism

public Integer getParallelism()

getBootstrapServers

public String[] getBootstrapServers()

setBootstrapServers

public void setBootstrapServers(String... bootstrapServers)

getElementGenerator

public Class<? extends Function<Iterable<?>,Iterable<? extends Element>>> getElementGenerator()

setElementGenerator

public void setElementGenerator(Class<? extends Function<Iterable<?>,Iterable<? extends Element>>> elementGenerator)

isValidate

public boolean isValidate()

Specified by:

isValidate in interface Validatable

Returns:

true if the operation should be validated. Otherwise false.

setValidate

public void setValidate(boolean validate)

Specified by:

setValidate in interface Validatable

Parameters:

validate - true if the operation should be validated. Otherwise false.

isSkipInvalidElements

public boolean isSkipInvalidElements()

Specified by:

isSkipInvalidElements in interface Validatable

Returns:

true if invalid elements should be skipped. Otherwise false if the operation should fail.

setSkipInvalidElements

public void setSkipInvalidElements(boolean skipInvalidElements)

Specified by:

setSkipInvalidElements in interface Validatable

Parameters:

skipInvalidElements - true if invalid elements should be skipped. Otherwise false if the operation should fail.

getConsumeAs

public Class<?> getConsumeAs()

setConsumeAs

public void setConsumeAs(Class<?> consumeAs)

getOptions

public Map<String,String> getOptions()

Specified by:

getOptions in interface Operation

Returns:

the operation options. This may contain store specific options such as authorisation strings or and other properties required for the operation to be executed. Note these options will probably not be interpreted in the same way by every store implementation.

setOptions

public void setOptions(Map<String,String> options)

Specified by:

setOptions in interface Operation

Parameters:

options - the operation options. This may contain store specific options such as authorisation strings or and other properties required for the operation to be executed. Note these options will probably not be interpreted in the same way by every store implementation.

validate

public uk.gov.gchq.koryphe.ValidationResult validate()

Description copied from interface: Operation

Validates an operation. This should be used to validate that fields have been be configured correctly. By default no validation is applied. Override this method to implement validation.

Specified by:

validate in interface Operation

Returns:

validation result.

shallowClone

public AddElementsFromKafka shallowClone()

Description copied from interface: Operation

Operation implementations should ensure a ShallowClone method is implemented. Performs a shallow clone. Creates a new instance and copies the fields across. It does not clone the fields. If the operation contains nested operations, these must also be cloned.

Specified by:

shallowClone in interface Operation

Returns:

shallow clone