public class GridGgfsHadoopFileSystem extends FileSystem
GGFS Hadoop 1.x file system driver over file system API. To use
GGFS as Hadoop file system, you should configure this class
in Hadoop's core-site.xml as follows:
<property>
<name>fs.default.name</name>
<value>ggfs://ipc</value>
</property>
<property>
<name>fs.ggfs.impl</name>
<value>org.gridgain.grid.ggfs.hadoop.GridGgfsHadoopFileSystem</value>
</property>
You should also add GridGain JAR and all libraries to Hadoop classpath. To
do this, add following lines to conf/hadoop-env.sh script in Hadoop
distribution:
export GRIDGAIN_HOME=/path/to/GridGain/distribution export HADOOP_CLASSPATH=$GRIDGAIN_HOME/gridgain*.jar for f in $GRIDGAIN_HOME/libs/*.jar; do export HADOOP_CLASSPATH=$HADOOP_CLASSPATH:$f; done
job-submitter or job-scheduler processes, while data nodes are usually
started together with Hadoop task-tracker processes.
For sample client and data node configuration refer to config/hadoop/default-config-client.xml
and config/hadoop/default-config.xml configuration files in GridGain installation.
FileSystem.Statistics| Modifier and Type | Field and Description |
|---|---|
static FileStatus[] |
EMPTY_FILE_STATUS
Empty array of file statuses.
|
static String |
GGFS_MANAGEMENT
Internal property to indicate management connection.
|
DEFAULT_FS, FS_DEFAULT_NAME_KEY, LOG, SHUTDOWN_HOOK_PRIORITY, statistics| Constructor and Description |
|---|
GridGgfsHadoopFileSystem() |
| Modifier and Type | Method and Description |
|---|---|
FSDataOutputStream |
append(Path f,
int bufSize,
Progressable progress) |
void |
close() |
void |
colocateFileWrites(boolean colocateFileWrites)
Public setter that can be used by direct users of FS or Visor.
|
FSDataOutputStream |
create(Path f,
FsPermission perm,
boolean overwrite,
int bufSize,
short replication,
long blockSize,
Progressable progress) |
boolean |
delete(Path f) |
boolean |
delete(Path f,
boolean recursive) |
ContentSummary |
getContentSummary(Path f) |
long |
getDefaultBlockSize() |
short |
getDefaultReplication() |
BlockLocation[] |
getFileBlockLocations(FileStatus status,
long start,
long len) |
FileStatus |
getFileStatus(Path f) |
URI |
getUri() |
Path |
getWorkingDirectory() |
void |
initialize(URI name,
Configuration cfg) |
FileStatus[] |
listStatus(Path f) |
boolean |
mkdirs(Path f,
FsPermission perm) |
GridGgfsMode |
mode(GridGgfsPath path)
Resolve path mode.
|
GridGgfsMode |
mode(Path path)
Resolve path mode.
|
FSDataInputStream |
open(Path f,
int bufSize) |
boolean |
rename(Path src,
Path dst) |
void |
setOwner(Path p,
String username,
String grpName) |
void |
setPermission(Path p,
FsPermission perm) |
void |
setTimes(Path p,
long mtime,
long atime) |
void |
setWorkingDirectory(Path newPath) |
String |
toString() |
addDelegationTokens, append, append, cancelDeleteOnExit, checkPath, clearStatistics, closeAll, closeAllForUGI, completeLocalOutput, concat, copyFromLocalFile, copyFromLocalFile, copyFromLocalFile, copyFromLocalFile, copyToLocalFile, copyToLocalFile, copyToLocalFile, create, create, create, create, create, create, create, create, create, create, create, create, createNewFile, createNonRecursive, createNonRecursive, createNonRecursive, deleteOnExit, exists, get, get, get, getAllStatistics, getBlockSize, getCanonicalServiceName, getCanonicalUri, getChildFileSystems, getDefaultBlockSize, getDefaultPort, getDefaultReplication, getDefaultUri, getDelegationToken, getFileBlockLocations, getFileChecksum, getFileSystemClass, getHomeDirectory, getInitialWorkingDirectory, getLength, getLocal, getName, getNamed, getReplication, getScheme, getServerDefaults, getServerDefaults, getStatistics, getStatistics, getStatus, getStatus, getUsed, globStatus, globStatus, isDirectory, isFile, listCorruptFileBlocks, listFiles, listLocatedStatus, listLocatedStatus, listStatus, listStatus, listStatus, makeQualified, mkdirs, mkdirs, moveFromLocalFile, moveFromLocalFile, moveToLocalFile, newInstance, newInstance, newInstance, newInstanceLocal, open, primitiveCreate, primitiveMkdir, primitiveMkdir, printStatistics, processDeleteOnExit, rename, resolvePath, setDefaultUri, setDefaultUri, setReplication, setVerifyChecksum, setWriteChecksum, startLocalOutputgetConf, setConfpublic static final String GGFS_MANAGEMENT
public static final FileStatus[] EMPTY_FILE_STATUS
public URI getUri()
getUri in class FileSystempublic void colocateFileWrites(boolean colocateFileWrites)
colocateFileWrites - Whether all ongoing file writes should be colocated.public void initialize(URI name, Configuration cfg) throws IOException
initialize in class FileSystemIOExceptionpublic short getDefaultReplication()
getDefaultReplication in class FileSystempublic void close()
throws IOException
close in interface Closeableclose in interface AutoCloseableclose in class FileSystemIOExceptionpublic void setTimes(Path p, long mtime, long atime) throws IOException
setTimes in class FileSystemIOExceptionpublic void setPermission(Path p, FsPermission perm) throws IOException
setPermission in class FileSystemIOExceptionpublic void setOwner(Path p, String username, String grpName) throws IOException
setOwner in class FileSystemIOExceptionpublic FSDataInputStream open(Path f, int bufSize) throws IOException
open in class FileSystemIOExceptionpublic FSDataOutputStream create(Path f, FsPermission perm, boolean overwrite, int bufSize, short replication, long blockSize, Progressable progress) throws IOException
create in class FileSystemIOExceptionpublic FSDataOutputStream append(Path f, int bufSize, Progressable progress) throws IOException
append in class FileSystemIOExceptionpublic boolean rename(Path src, Path dst) throws IOException
rename in class FileSystemIOExceptionpublic boolean delete(Path f) throws IOException
delete in class FileSystemIOExceptionpublic boolean delete(Path f, boolean recursive) throws IOException
delete in class FileSystemIOExceptionpublic FileStatus[] listStatus(Path f) throws IOException
listStatus in class FileSystemIOExceptionpublic void setWorkingDirectory(Path newPath)
setWorkingDirectory in class FileSystempublic Path getWorkingDirectory()
getWorkingDirectory in class FileSystempublic boolean mkdirs(Path f, FsPermission perm) throws IOException
mkdirs in class FileSystemIOExceptionpublic FileStatus getFileStatus(Path f) throws IOException
getFileStatus in class FileSystemIOExceptionpublic ContentSummary getContentSummary(Path f) throws IOException
getContentSummary in class FileSystemIOExceptionpublic BlockLocation[] getFileBlockLocations(FileStatus status, long start, long len) throws IOException
getFileBlockLocations in class FileSystemIOExceptionpublic long getDefaultBlockSize()
getDefaultBlockSize in class FileSystempublic GridGgfsMode mode(Path path)
path - HDFS path.public GridGgfsMode mode(GridGgfsPath path)
path - GGFS path.
Core Products
:
Data Grid ver. 6.0.2,
Streaming ver. 6.0.2
Accelerators
:
Hadoop Accelerator ver. 6.0.2
Release Date
:
March 24 2014