Package uk.gov.gchq.gaffer.spark

Class SparkContextUtil

java.lang.Object

uk.gov.gchq.gaffer.spark.SparkContextUtil

public final class SparkContextUtil
extends Object

A SparkContextUtil is a utility class for adding and retrieving a SparkSession to/from the user Context.

Field Summary

Fields

Modifier and Type	Field	Description
static String	SPARK_CONTEXT_CONFIG_KEY

Method Summary

Modifier and Type	Method	Description
static void	addSparkSession(Context context, org.apache.spark.sql.SparkSession sparkSession)	Adds a spark session to the given Context.
static void	addSparkSession(Context context, StoreProperties storeProperties)	Adds a spark session to the given Context.
static Context	createContext(User user, org.apache.spark.sql.SparkSession sparkSession)	Create a new Context with the given user and spark session.
static Context	createContext(User user, StoreProperties storeProperties)	Create a new Context with the given user and store properties
static org.apache.spark.sql.SparkSession	createSparkSession(StoreProperties storeProperties)
static org.apache.spark.sql.SparkSession	getSparkSession(Context context, StoreProperties properties)	Extracts the SparkSession from the Context.

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

SPARK_CONTEXT_CONFIG_KEY

public static final String SPARK_CONTEXT_CONFIG_KEY

See Also:

Constant Field Values

Method Detail

addSparkSession

public static void addSparkSession(Context context,
                                   StoreProperties storeProperties)

Adds a spark session to the given Context.

Parameters:

context - the user context

storeProperties - the store properties - used to create a spark session

addSparkSession

public static void addSparkSession(Context context,
                                   org.apache.spark.sql.SparkSession sparkSession)

Adds a spark session to the given Context.

Parameters:

context - the user context

sparkSession - the spark session to add to the context

createContext

public static Context createContext(User user,
                                    StoreProperties storeProperties)

Create a new Context with the given user and store properties

Parameters:

user - the user

storeProperties - the store properties

Returns:

the new Context.

createContext

public static Context createContext(User user,
                                    org.apache.spark.sql.SparkSession sparkSession)

Create a new Context with the given user and spark session.

Parameters:

user - the user

sparkSession - the spark session

Returns:

the new Context.

getSparkSession

public static org.apache.spark.sql.SparkSession getSparkSession(Context context,
                                                                StoreProperties properties)

Extracts the SparkSession from the Context. If there is no SparkSession in the Context then a new SparkSession instance is created and added to the context.

Parameters:

context - the User Context

properties - the store properties - used to create a spark session if required

Returns:

the SparkSession

createSparkSession

public static org.apache.spark.sql.SparkSession createSparkSession(StoreProperties storeProperties)