public class GridGgfsHadoopParameters extends Object
core-site.xml file
or passed to map-reduce task directly when using Hadoop driver for GGFS file system:
fs.ggfs.[name].endpoint.type - this parameter specifies IPC endpoint type. In case not provided,
"tcp" will be used for Windows platforms and "shmem" for all the other platforms.
fs.ggfs.[name].endpoint.host - optional name of loopback interface to which IPC endpoint is bound.
In case not provided, IPC endpoint will be bound to 127.0.0.1 IP address by default.
fs.ggfs.[name].endpoint.port - this parameter specifies IPC endpoint port. In case not provided,
default port GridGgfsConfiguration.DFLT_IPC_PORT will be used.
fs.ggfs.[name].open.sequential_reads_before_prefetch - this parameter overrides
the one specified in GridGgfsConfiguration.getSequentialReadsBeforePrefetch()
GGFS data node configuration property.
fs.ggfs.[name].log.enabled - specifies whether GGFS sampling logger is enabled. If
true, then all file system operations will be logged to a file.
fs.ggfs.[name].log.dir - specifies log directory where sampling log files should be placed.fs.ggfs.[name].log.batch_size - specifies how many log entries are accumulated in a batch before
it gets flushed to log file. Higher values will imply greater performance, but will increase delay
before record appears in the log file.
fs.ggfs.[name].colocated.writes - specifies whether written files should be colocated on data
node to which client is connected. If true, file will not be distributed and will be written
to a single data node. Default value is true.
[name] is arbitrary file system name which you specify in file system URI authority part. E.g. in
case your file system URI is ggfs://ipc then name will be ipc.
Sample configuration that can be placed to core-site.xml file:
<property>
<name>fs.ggfs.ipc.endpoint.type</name>
<value>shmem</value>
</property>
<property>
<name>fs.ggfs.ipc.endpoint.port</name>
<value>10501</value>
</property>
<property>
<name>fs.ggfs.ipc.log.enabled</name>
<value>true</value>
</property>
<property>
<name>fs.ggfs.ipc.log.dir</name>
<value>/home/gridgain/log/sampling</value>
</property>
<property>
<name>fs.ggfs.ipc.log.batch_size</name>
<value>16</value>
</property>
Parameters could also be specified per mapreduce job, e.g.
hadoop jar myjarfile.jar MyMapReduceJob -Dfs.ggfs.open.sequential_reads_before_prefetch=4If you want to use these parameters in code, then you have to substitute you file system name in it. The easiest way to do that is
String.format(PARAM_GGFS_ENDPOINT_TYPE, [name]).| Modifier and Type | Field and Description |
|---|---|
static String |
PARAM_GGFS_COLOCATED_WRITES
Parameter name for control over file colocation write mode.
|
static String |
PARAM_GGFS_ENDPOINT_HOST
Parameter name for IPC endpoint host.
|
static String |
PARAM_GGFS_ENDPOINT_PORT
Parameter name for IPC endpoint port.
|
static String |
PARAM_GGFS_ENDPOINT_TYPE
Parameter name for IPC endpoint type.
|
static String |
PARAM_GGFS_LOG_BATCH_SIZE
Parameter name for log batch size.
|
static String |
PARAM_GGFS_LOG_DIR
Parameter name for client logger directory.
|
static String |
PARAM_GGFS_LOG_ENABLED
Parameter name for log enabled flag.
|
static String |
PARAM_GGFS_SEQ_READS_BEFORE_PREFETCH
Parameter name for custom sequential reads before prefetch value.
|
| Constructor and Description |
|---|
GridGgfsHadoopParameters() |
public static final String PARAM_GGFS_ENDPOINT_TYPE
public static final String PARAM_GGFS_ENDPOINT_HOST
public static final String PARAM_GGFS_ENDPOINT_PORT
public static final String PARAM_GGFS_COLOCATED_WRITES
public static final String PARAM_GGFS_SEQ_READS_BEFORE_PREFETCH
public static final String PARAM_GGFS_LOG_DIR
public static final String PARAM_GGFS_LOG_BATCH_SIZE
public static final String PARAM_GGFS_LOG_ENABLED
Core Products
:
Data Grid ver. 6.0.2,
Streaming ver. 6.0.2
Accelerators
:
Hadoop Accelerator ver. 6.0.2
Release Date
:
March 24 2014