public class GridGgfsHadoopFileSystem extends AbstractFileSystem implements Closeable
GGFS Hadoop 2.x file system driver over file system API. To use
GGFS as Hadoop file system, you should configure this class
in Hadoop's core-site.xml as follows:
<property>
<name>fs.default.name</name>
<value>ggfs://ipc</value>
</property>
<property>
<name>fs.ggfs.impl</name>
<value>org.gridgain.grid.ggfs.hadoop.GridGgfsHadoopFileSystem</value>
</property>
You should also add GridGain JAR and all libraries to Hadoop classpath. To
do this, add following lines to conf/hadoop-env.sh script in Hadoop
distribution:
export GRIDGAIN_HOME=/path/to/GridGain/distribution export HADOOP_CLASSPATH=$GRIDGAIN_HOME/gridgain*.jar for f in $GRIDGAIN_HOME/libs/*.jar; do export HADOOP_CLASSPATH=$HADOOP_CLASSPATH:$f; done
job-submitter or job-scheduler processes, while data nodes are usually
started together with Hadoop task-tracker processes.
For sample client and data node configuration refer to config/hadoop/default-config-client.xml
and config/hadoop/default-config.xml configuration files in GridGain installation.
statistics| Constructor and Description |
|---|
GridGgfsHadoopFileSystem(URI name,
Configuration cfg) |
| Modifier and Type | Method and Description |
|---|---|
void |
close() |
void |
colocateFileWrites(boolean colocateFileWrites)
Public setter that can be used by direct users of FS or Visor.
|
FSDataOutputStream |
createInternal(Path f,
EnumSet<CreateFlag> flag,
FsPermission perm,
int bufSize,
short replication,
long blockSize,
Progressable progress,
Options.ChecksumOpt checksumOpt,
boolean createParent) |
boolean |
delete(Path f,
boolean recursive) |
BlockLocation[] |
getFileBlockLocations(Path path,
long start,
long len) |
FileChecksum |
getFileChecksum(Path f) |
FileStatus |
getFileStatus(Path f) |
FsStatus |
getFsStatus() |
FsServerDefaults |
getServerDefaults() |
int |
getUriDefaultPort() |
FileStatus[] |
listStatus(Path f) |
void |
mkdir(Path f,
FsPermission perm,
boolean createParent) |
GridGgfsMode |
mode(Path path)
Resolve path mode.
|
FSDataInputStream |
open(Path f,
int bufSize) |
void |
renameInternal(Path src,
Path dst) |
void |
setOwner(Path p,
String usr,
String grp) |
void |
setPermission(Path p,
FsPermission perm) |
boolean |
setReplication(Path f,
short replication) |
void |
setTimes(Path f,
long mtime,
long atime) |
void |
setVerifyChecksum(boolean verifyChecksum) |
boolean |
supportsSymlinks() |
String |
toString() |
checkPath, checkScheme, clearStatistics, create, createFileSystem, createSymlink, equals, get, getAllStatistics, getCanonicalServiceName, getDelegationTokens, getFileLinkStatus, getFsStatus, getHomeDirectory, getInitialWorkingDirectory, getLinkTarget, getStatistics, getStatistics, getUri, getUriPath, hashCode, listCorruptFileBlocks, listLocatedStatus, listStatusIterator, makeQualified, open, printStatistics, rename, renameInternal, resolvePathpublic GridGgfsHadoopFileSystem(URI name, Configuration cfg) throws URISyntaxException, IOException
name - URI for file system.cfg - Configuration.URISyntaxException - if name has invalid syntax.IOException - If initialization failed.public void colocateFileWrites(boolean colocateFileWrites)
colocateFileWrites - Whether all ongoing file writes should be colocated.public void close()
throws IOException
close in interface Closeableclose in interface AutoCloseableIOExceptionpublic int getUriDefaultPort()
getUriDefaultPort in class AbstractFileSystempublic FsServerDefaults getServerDefaults() throws IOException
getServerDefaults in class AbstractFileSystemIOExceptionpublic boolean setReplication(Path f, short replication) throws AccessControlException, FileNotFoundException, UnresolvedLinkException, IOException
setReplication in class AbstractFileSystemAccessControlExceptionFileNotFoundExceptionUnresolvedLinkExceptionIOExceptionpublic void setTimes(Path f, long mtime, long atime) throws AccessControlException, FileNotFoundException, UnresolvedLinkException, IOException
setTimes in class AbstractFileSystemAccessControlExceptionFileNotFoundExceptionUnresolvedLinkExceptionIOExceptionpublic FsStatus getFsStatus() throws IOException
getFsStatus in class AbstractFileSystemIOExceptionpublic void setPermission(Path p, FsPermission perm) throws IOException
setPermission in class AbstractFileSystemIOExceptionpublic void setOwner(Path p, String usr, String grp) throws IOException
setOwner in class AbstractFileSystemIOExceptionpublic FSDataInputStream open(Path f, int bufSize) throws IOException
open in class AbstractFileSystemIOExceptionpublic FSDataOutputStream createInternal(Path f, EnumSet<CreateFlag> flag, FsPermission perm, int bufSize, short replication, long blockSize, Progressable progress, Options.ChecksumOpt checksumOpt, boolean createParent) throws AccessControlException, FileAlreadyExistsException, ParentNotDirectoryException, UnsupportedFileSystemException, UnresolvedLinkException, IOException
public boolean supportsSymlinks()
supportsSymlinks in class AbstractFileSystempublic void renameInternal(Path src, Path dst) throws IOException
renameInternal in class AbstractFileSystemIOExceptionpublic boolean delete(Path f, boolean recursive) throws IOException
delete in class AbstractFileSystemIOExceptionpublic void setVerifyChecksum(boolean verifyChecksum)
throws AccessControlException,
IOException
setVerifyChecksum in class AbstractFileSystemAccessControlExceptionIOExceptionpublic FileChecksum getFileChecksum(Path f) throws IOException
getFileChecksum in class AbstractFileSystemIOExceptionpublic FileStatus[] listStatus(Path f) throws IOException
listStatus in class AbstractFileSystemIOExceptionpublic void mkdir(Path f, FsPermission perm, boolean createParent) throws IOException
mkdir in class AbstractFileSystemIOExceptionpublic FileStatus getFileStatus(Path f) throws IOException
getFileStatus in class AbstractFileSystemIOExceptionpublic BlockLocation[] getFileBlockLocations(Path path, long start, long len) throws IOException
getFileBlockLocations in class AbstractFileSystemIOExceptionpublic GridGgfsMode mode(Path path)
path - HDFS path.
Core Products
:
Data Grid ver. 6.0.2,
Streaming ver. 6.0.2
Accelerators
:
Hadoop Accelerator ver. 6.0.2
Release Date
:
March 24 2014