org.apache.ignite.igfs.mapreduce

Interface IgfsRecordResolver

All Superinterfaces:

Serializable

All Known Implementing Classes:

IgfsByteDelimiterRecordResolver, IgfsFixedLengthRecordResolver, IgfsNewLineRecordResolver, IgfsStringDelimiterRecordResolver

public interface IgfsRecordResolver
extends Serializable

IGFS record resolver. When IgfsTask is split into IgfsJobs each produced job will obtain IgfsFileRange based on file data location. Record resolver is invoked in each job before actual execution in order to adjust record boundaries in a way consistent with user data.

E.g., you may want to split your task into jobs so that each job process zero, one or several lines from that file. But file is split into ranges based on block locations, not new line boundaries. Using convenient record resolver you can adjust job range so that it covers the whole line(s).

The following record resolvers are available out of the box:

• IgfsFixedLengthRecordResolver
• IgfsByteDelimiterRecordResolver
• IgfsStringDelimiterRecordResolver
• IgfsNewLineRecordResolver

Method Summary

Modifier and Type	Method and Description
@Nullable IgfsFileRange	resolveRecords(IgniteFileSystem fs, IgfsInputStream stream, IgfsFileRange suggestedRecord) Adjusts record start offset and length.

Method Detail

resolveRecords

@Nullable
@Nullable IgfsFileRange resolveRecords(IgniteFileSystem fs,
                                                 IgfsInputStream stream,
                                                 IgfsFileRange suggestedRecord)
                                          throws IgniteException,
                                                 IOException

Adjusts record start offset and length.

Parameters:

fs - IGFS instance to use.

stream - Input stream for split file.

suggestedRecord - Suggested file system record.

Returns:

New adjusted record. If this method returns null, original record is ignored.

Throws:

IgniteException - If resolve failed.

IOException - If resolve failed.