C
- Type of a partition context
.D
- Type of a partition data
.public interface Dataset<C extends Serializable,D extends AutoCloseable> extends AutoCloseable
context
(reliably
stored segment) and data
(unreliably stored segment, which can be recovered from an upstream data and a
context
if needed). Computations are performed in a MapReduce
manner, what allows to reduce a
network traffic for most of the machine learning algorithms.
Dataset functionality allows to implement iterative machine learning algorithms via introducing computation
context. In case iterative algorithm requires to maintain a state available and updatable on every iteration this
state can be stored in the context
of the partition and after that it will be available in further
computations even if the Ignite Cache partition will be moved to another node because of node failure or rebalancing.
Partition context
should be Serializable
to be saved in Ignite Cache. Partition data
should be AutoCloseable
to allow system to clean up correspondent resources when partition data
is
not needed anymore.
CacheBasedDataset
,
LocalDataset
,
DatasetFactory
Modifier and Type | Method and Description |
---|---|
default void |
compute(IgniteBiConsumer<D,LearningEnvironment> map)
|
default <R> R |
compute(IgniteBiFunction<D,LearningEnvironment,R> map,
IgniteBinaryOperator<R> reduce)
Applies the specified
map function to every partition data and LearningEnvironment
in the dataset and then reduces map results to final result by using the reduce function. |
<R> R |
compute(IgniteBiFunction<D,LearningEnvironment,R> map,
IgniteBinaryOperator<R> reduce,
R identity)
Applies the specified
map function to every partition data and LearningEnvironment
in the dataset and then reduces map results to final result by using the reduce function. |
default void |
compute(IgniteConsumer<D> map)
Applies the specified
map function to every partition data in the dataset. |
default <R> R |
compute(IgniteFunction<D,R> map,
IgniteBinaryOperator<R> reduce)
Applies the specified
map function to every partition data in the dataset and then reduces
map results to final result by using the reduce function. |
default <R> R |
compute(IgniteFunction<D,R> map,
IgniteBinaryOperator<R> reduce,
R identity)
Applies the specified
map function to every partition data in the dataset and then reduces
map results to final result by using the reduce function. |
default void |
computeWithCtx(IgniteBiConsumer<C,D> map)
Applies the specified
map function to every partition data and context in the dataset. |
default <R> R |
computeWithCtx(IgniteBiFunction<C,D,R> map,
IgniteBinaryOperator<R> reduce)
Applies the specified
map function to every partition data and context in the dataset
and then reduces map results to final result by using the reduce function. |
default <R> R |
computeWithCtx(IgniteBiFunction<C,D,R> map,
IgniteBinaryOperator<R> reduce,
R identity)
Applies the specified
map function to every partition data and context in the dataset
and then reduces map results to final result by using the reduce function. |
default void |
computeWithCtx(IgniteTriConsumer<C,D,LearningEnvironment> map)
Applies the specified
map function to every partition data , context and
LearningEnvironment in the dataset. |
default <R> R |
computeWithCtx(IgniteTriFunction<C,D,LearningEnvironment,R> map,
IgniteBinaryOperator<R> reduce)
Applies the specified
map function to every partition data , context and
LearningEnvironment in the dataset and then reduces map results to final
result by using the reduce function. |
<R> R |
computeWithCtx(IgniteTriFunction<C,D,LearningEnvironment,R> map,
IgniteBinaryOperator<R> reduce,
R identity)
Applies the specified
map function to every partition data , context and partition
index in the dataset and then reduces map results to final result by using the reduce function. |
default <I extends Dataset<C,D>> |
wrap(IgniteFunction<Dataset<C,D>,I> wrapper)
Wraps this dataset into the specified wrapper to introduce new functionality based on
compute and
computeWithCtx methods. |
close
<R> R computeWithCtx(IgniteTriFunction<C,D,LearningEnvironment,R> map, IgniteBinaryOperator<R> reduce, R identity)
map
function to every partition data
, context
and partition
index in the dataset and then reduces map
results to final result by using the reduce
function.R
- Type of a result.map
- Function applied to every partition data
, context
and LearningEnvironment
.reduce
- Function applied to results of map
to get final result.identity
- Identity.<R> R compute(IgniteBiFunction<D,LearningEnvironment,R> map, IgniteBinaryOperator<R> reduce, R identity)
map
function to every partition data
and LearningEnvironment
in the dataset and then reduces map
results to final result by using the reduce
function.R
- Type of a result.map
- Function applied to every partition data
and LearningEnvironment
.reduce
- Function applied to results of map
to get final result.identity
- Identity.default <R> R computeWithCtx(IgniteTriFunction<C,D,LearningEnvironment,R> map, IgniteBinaryOperator<R> reduce)
map
function to every partition data
, context
and
LearningEnvironment
in the dataset and then reduces map
results to final
result by using the reduce
function.R
- Type of a result.map
- Function applied to every partition data
, context
and LearningEnvironment
.reduce
- Function applied to results of map
to get final result.default <R> R compute(IgniteBiFunction<D,LearningEnvironment,R> map, IgniteBinaryOperator<R> reduce)
map
function to every partition data
and LearningEnvironment
in the dataset and then reduces map
results to final result by using the reduce
function.R
- Type of a result.map
- Function applied to every partition data
and LearningEnvironment
.reduce
- Function applied to results of map
to get final result.default <R> R computeWithCtx(IgniteBiFunction<C,D,R> map, IgniteBinaryOperator<R> reduce, R identity)
map
function to every partition data
and context
in the dataset
and then reduces map
results to final result by using the reduce
function.R
- Type of a result.map
- Function applied to every partition data
and context
.reduce
- Function applied to results of map
to get final result.identity
- Identity.default <R> R compute(IgniteFunction<D,R> map, IgniteBinaryOperator<R> reduce, R identity)
map
function to every partition data
in the dataset and then reduces
map
results to final result by using the reduce
function.R
- Type of a result.map
- Function applied to every partition data
.reduce
- Function applied to results of map
to get final result.identity
- Identity.default <R> R computeWithCtx(IgniteBiFunction<C,D,R> map, IgniteBinaryOperator<R> reduce)
map
function to every partition data
and context
in the dataset
and then reduces map
results to final result by using the reduce
function.R
- Type of a result.map
- Function applied to every partition data
and context
.reduce
- Function applied to results of map
to get final result.default <R> R compute(IgniteFunction<D,R> map, IgniteBinaryOperator<R> reduce)
map
function to every partition data
in the dataset and then reduces
map
results to final result by using the reduce
function.R
- Type of a result.map
- Function applied to every partition data
.reduce
- Function applied to results of map
to get final result.default void computeWithCtx(IgniteTriConsumer<C,D,LearningEnvironment> map)
map
function to every partition data
, context
and
LearningEnvironment
in the dataset.map
- Function applied to every partition data
, context
and partition index.default void compute(IgniteBiConsumer<D,LearningEnvironment> map)
map
- Function applied to every partition data
and partition index.default void computeWithCtx(IgniteBiConsumer<C,D> map)
map
function to every partition data
and context
in the dataset.map
- Function applied to every partition data
and context
.default void compute(IgniteConsumer<D> map)
map
function to every partition data
in the dataset.map
- Function applied to every partition data
.default <I extends Dataset<C,D>> I wrap(IgniteFunction<Dataset<C,D>,I> wrapper)
compute
and
computeWithCtx
methods.I
- Type of a new wrapped dataset.wrapper
- Dataset wrapper.
GridGain In-Memory Computing Platform : ver. 8.9.14 Release Date : November 5 2024