org.apache.ignite.configuration

Class HadoopConfiguration

java.lang.Object

org.apache.ignite.configuration.HadoopConfiguration

public class HadoopConfiguration
extends Object

Ignite Hadoop Accelerator configuration.

Field Summary

Fields

Modifier and Type	Field and Description
static boolean	DFLT_EXTERNAL_EXECUTION Default value for external execution flag.
static long	DFLT_FINISHED_JOB_INFO_TTL Default finished job info time-to-live.
static int	DFLT_MAX_PARALLEL_TASKS Default value for the max parallel tasks.
static int	DFLT_MAX_TASK_QUEUE_SIZE Default value for the max task queue size.

Constructor Summary

Constructors

Constructor and Description
HadoopConfiguration() Default constructor.
HadoopConfiguration(HadoopConfiguration cfg) Copy constructor.

Method Summary

Modifier and Type	Method and Description
long	getFinishedJobInfoTtl() Gets finished job info time-to-live in milliseconds.
HadoopMapReducePlanner	getMapReducePlanner() Gets Hadoop map-reduce planner, a component which defines job execution plan based on job configuration and current grid topology.
int	getMaxParallelTasks() Gets max number of local tasks that may be executed in parallel.
int	getMaxTaskQueueSize() Gets max task queue size.
@Nullable String[]	getNativeLibraryNames() Get native library names.
HadoopConfiguration	setFinishedJobInfoTtl(long finishedJobInfoTtl) Sets finished job info time-to-live.
HadoopConfiguration	setMapReducePlanner(HadoopMapReducePlanner planner) Sets Hadoop map-reduce planner, a component which defines job execution plan based on job configuration and current grid topology.
HadoopConfiguration	setMaxParallelTasks(int maxParallelTasks) Sets max number of local tasks that may be executed in parallel.
HadoopConfiguration	setMaxTaskQueueSize(int maxTaskQueueSize) Sets max task queue size.
HadoopConfiguration	setNativeLibraryNames(String... libNames) Set native library names.
String	toString()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

DFLT_FINISHED_JOB_INFO_TTL

public static final long DFLT_FINISHED_JOB_INFO_TTL

Default finished job info time-to-live.

See Also:

Constant Field Values

DFLT_EXTERNAL_EXECUTION

public static final boolean DFLT_EXTERNAL_EXECUTION

Default value for external execution flag.

See Also:

Constant Field Values

DFLT_MAX_PARALLEL_TASKS

public static final int DFLT_MAX_PARALLEL_TASKS

Default value for the max parallel tasks.

DFLT_MAX_TASK_QUEUE_SIZE

public static final int DFLT_MAX_TASK_QUEUE_SIZE

Default value for the max task queue size.

See Also:

Constant Field Values

Constructor Detail

HadoopConfiguration

public HadoopConfiguration()

Default constructor.

HadoopConfiguration

public HadoopConfiguration(HadoopConfiguration cfg)

Copy constructor.

Parameters:

cfg - Configuration to copy.

Method Detail

getMaxParallelTasks

public int getMaxParallelTasks()

Gets max number of local tasks that may be executed in parallel.

Returns:

Max number of local tasks that may be executed in parallel.

setMaxParallelTasks

public HadoopConfiguration setMaxParallelTasks(int maxParallelTasks)

Sets max number of local tasks that may be executed in parallel.

Parameters:

maxParallelTasks - Max number of local tasks that may be executed in parallel.

Returns:

this for chaining.

getMaxTaskQueueSize

public int getMaxTaskQueueSize()

Gets max task queue size.

Returns:

Max task queue size.

setMaxTaskQueueSize

public HadoopConfiguration setMaxTaskQueueSize(int maxTaskQueueSize)

Sets max task queue size.

Parameters:

maxTaskQueueSize - Max task queue size.

Returns:

this for chaining.

getFinishedJobInfoTtl

public long getFinishedJobInfoTtl()

Gets finished job info time-to-live in milliseconds.

Returns:

Finished job info time-to-live.

setFinishedJobInfoTtl

public HadoopConfiguration setFinishedJobInfoTtl(long finishedJobInfoTtl)

Sets finished job info time-to-live.

Parameters:

finishedJobInfoTtl - Finished job info time-to-live.

Returns:

this for chaining.

getMapReducePlanner

public HadoopMapReducePlanner getMapReducePlanner()

Gets Hadoop map-reduce planner, a component which defines job execution plan based on job configuration and current grid topology.

Returns:

Map-reduce planner.

setMapReducePlanner

public HadoopConfiguration setMapReducePlanner(HadoopMapReducePlanner planner)

Sets Hadoop map-reduce planner, a component which defines job execution plan based on job configuration and current grid topology.

Parameters:

planner - Map-reduce planner.

Returns:

this for chaining.

getNativeLibraryNames

@Nullable
public @Nullable String[] getNativeLibraryNames()

Get native library names.

Ignite Hadoop Accelerator executes all Hadoop jobs and tasks in the same process, isolating them with help of classloaders. If Hadoop job or task loads a native library, it might lead to exception, because Java do not allow to load the same library multiple times from different classloaders. To overcome the problem, you should to the following:

• Load necessary libraries in advance from base classloader; LifecycleBean is a good candidate for this;
• Add names of loaded libraries to this property, so that Hadoop engine is able to link them;
• Remove System.load(String) and System.loadLibrary(String) calls from your job/task.
*

Returns:

Native library names.

setNativeLibraryNames

public HadoopConfiguration setNativeLibraryNames(@Nullable
                                                 String... libNames)

Set native library names. See getNativeLibraryNames() for more information.

Parameters:

libNames - Native library names.

Returns:

this for chaining.

toString

public String toString()

Overrides:

toString in class Object