org.apache.ignite.ml.dataset.impl.cache

Class CacheBasedDataset<K,V,C extends Serializable,D extends AutoCloseable>

java.lang.Object

org.apache.ignite.ml.dataset.impl.cache.CacheBasedDataset<K,V,C,D>

Type Parameters:

K - Type of a key in upstream data.

V - Type of a value in upstream data.

C - Type of a partition context.

D - Type of a partition data.

All Implemented Interfaces:

AutoCloseable, Dataset<C,D>

public class CacheBasedDataset<K,V,C extends Serializable,D extends AutoCloseable>
extends Object
implements Dataset<C,D>

An implementation of dataset based on Ignite Cache, which is used as upstream and as reliable storage for partition context as well.

Constructor Summary

Constructors

Constructor and Description

CacheBasedDataset(Ignite ignite, IgniteCache<K,V> upstreamCache, IgniteBiPredicate<K,V> filter, UpstreamTransformerBuilder upstreamTransformerBuilder, IgniteCache<Integer,C> datasetCache, LearningEnvironmentBuilder envBuilder, PartitionDataBuilder<K,V,C,D> partDataBuilder, UUID datasetId, boolean upstreamKeepBinary, LearningEnvironment localLearningEnv, int retriesCnt) Constructs a new instance of dataset based on Ignite Cache, which is used as upstream and as reliable storage for partition context as well.

Method Summary

Modifier and Type	Method and Description
void	close()
<R> R	compute(IgniteBiFunction<D,LearningEnvironment,R> map, IgniteBinaryOperator<R> reduce, R identity) Applies the specified map function to every partition data and LearningEnvironment in the dataset and then reduces map results to final result by using the reduce function.
<R> R	computeWithCtx(IgniteTriFunction<C,D,LearningEnvironment,R> map, IgniteBinaryOperator<R> reduce, R identity) Applies the specified map function to every partition data, context and partition index in the dataset and then reduces map results to final result by using the reduce function.
IgniteCache<Integer,C>	getDatasetCache()
IgniteCache<K,V>	getUpstreamCache()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.ignite.ml.dataset.Dataset

compute, compute, compute, compute, compute, computeWithCtx, computeWithCtx, computeWithCtx, computeWithCtx, computeWithCtx, wrap

Constructor Detail

CacheBasedDataset

public CacheBasedDataset(Ignite ignite,
                         IgniteCache<K,V> upstreamCache,
                         IgniteBiPredicate<K,V> filter,
                         UpstreamTransformerBuilder upstreamTransformerBuilder,
                         IgniteCache<Integer,C> datasetCache,
                         LearningEnvironmentBuilder envBuilder,
                         PartitionDataBuilder<K,V,C,D> partDataBuilder,
                         UUID datasetId,
                         boolean upstreamKeepBinary,
                         LearningEnvironment localLearningEnv,
                         int retriesCnt)

Constructs a new instance of dataset based on Ignite Cache, which is used as upstream and as reliable storage for partition context as well.

Parameters:

ignite - Ignite instance.

upstreamCache - Ignite Cache with upstream data.

filter - Filter for upstream data.

upstreamTransformerBuilder - Transformer of upstream data (see description in DatasetBuilder).

datasetCache - Ignite Cache with partition context.

partDataBuilder - Partition data builder.

datasetId - Dataset ID.

localLearningEnv - Local learning environment.

retriesCnt - Number of retries for the case when one of partitions not found on the node where computation is performed.

Method Detail

computeWithCtx

public <R> R computeWithCtx(IgniteTriFunction<C,D,LearningEnvironment,R> map,
                            IgniteBinaryOperator<R> reduce,
                            R identity)

Applies the specified map function to every partition data, context and partition index in the dataset and then reduces map results to final result by using the reduce function.

Specified by:

computeWithCtx in interface Dataset<C extends Serializable,D extends AutoCloseable>

Type Parameters:

R - Type of a result.

Parameters:

map - Function applied to every partition data, context and LearningEnvironment.

reduce - Function applied to results of map to get final result.

identity - Identity.

Returns:

Final result.

compute

public <R> R compute(IgniteBiFunction<D,LearningEnvironment,R> map,
                     IgniteBinaryOperator<R> reduce,
                     R identity)

Applies the specified map function to every partition data and LearningEnvironment in the dataset and then reduces map results to final result by using the reduce function.

Specified by:

compute in interface Dataset<C extends Serializable,D extends AutoCloseable>

Type Parameters:

R - Type of a result.

Parameters:

map - Function applied to every partition data and LearningEnvironment.

reduce - Function applied to results of map to get final result.

identity - Identity.

Returns:

Final result.

close

public void close()

Specified by:

close in interface AutoCloseable

getUpstreamCache

public IgniteCache<K,V> getUpstreamCache()

getDatasetCache

public IgniteCache<Integer,C> getDatasetCache()