org.apache.ignite.ml.util.generators

Interface DataStreamGenerator

All Known Implementing Classes:

GaussianMixtureDataStream, RegressionDataStream, RingsDataStream, TwoSeparableClassesDataStream

public interface DataStreamGenerator

Provides general interface for generation of pseudorandom vectors according to shape defined by logic of specific data stream generator.

Field Summary

Fields

Modifier and Type	Field and Description
static int	FILL_CACHE_BATCH_SIZE Size of batch for IgniteCache.putAll(Map).

Method Summary

Modifier and Type	Method and Description
default DatasetBuilder<Vector,Double>	asDatasetBuilder(int datasetSize, IgniteBiPredicate<Vector,Double> filter, int partitions) Convert first N values from stream to DatasetBuilder.
default DatasetBuilder<Vector,Double>	asDatasetBuilder(int datasetSize, IgniteBiPredicate<Vector,Double> filter, int partitions, UpstreamTransformerBuilder upstreamTransformerBuilder) Convert first N values from stream to DatasetBuilder.
default DatasetBuilder<Vector,Double>	asDatasetBuilder(int datasetSize, int partitions) Convert first N values from stream to DatasetBuilder.
default Map<Vector,Double>	asMap(int datasetSize) Convert first N values from stream to map.
default DataStreamGenerator	blur(RandomProducer rnd) Apply pseudorandom noize to vectors without labels mapping.
default <K> void	fillCacheWithCustomKey(int datasetSize, IgniteCache<K,LabeledVector<Double>> cache, Function<LabeledVector<Double>,K> keyMapper) Fills given cache with labeled vectors from this generator and user defined mapper from vectors to keys.
default void	fillCacheWithVecHashAsKey(int datasetSize, IgniteCache<Integer,LabeledVector<Double>> cache) Fills given cache with labeled vectors from this generator as values and their hashcodes as keys.
default void	fillCacheWithVecUUIDAsKey(int datasetSize, IgniteCache<UUID,LabeledVector<Double>> cache) Fills given cache with labeled vectors from this generator as values and random UUIDs as keys
Stream<LabeledVector<Double>>	labeled()
default Stream<LabeledVector<Double>>	labeled(IgniteFunction<Vector,Double> classifier)
default DataStreamGenerator	mapVectors(IgniteFunction<Vector,Vector> f) Apply user defined mapper to vectors stream without labels hiding.
default Stream<Vector>	unlabeled()

Field Detail

FILL_CACHE_BATCH_SIZE

static final int FILL_CACHE_BATCH_SIZE

Size of batch for IgniteCache.putAll(Map).

See Also:

Constant Field Values

Method Detail

labeled

Stream<LabeledVector<Double>> labeled()

Returns:

Stream of LabeledVector in according to dataset shape.

unlabeled

default Stream<Vector> unlabeled()

Returns:

Stream of unlabeled Vector in according to dataset shape.

labeled

default Stream<LabeledVector<Double>> labeled(IgniteFunction<Vector,Double> classifier)

Parameters:

classifier - User defined classifier for vectors stream.

Returns:

Stream of LabeledVector in according to dataset shape and user's classifier.

mapVectors

default DataStreamGenerator mapVectors(IgniteFunction<Vector,Vector> f)

Apply user defined mapper to vectors stream without labels hiding.

Parameters:

f - Mapper of vectors of data stream.

Returns:

Stream of mapped vectors.

blur

default DataStreamGenerator blur(RandomProducer rnd)

Apply pseudorandom noize to vectors without labels mapping. Such method can be useful in cases when vectors with different labels should be mixed between them on class bounds.

Parameters:

rnd - Generator of pseudorandom scalars modifying vector components with label saving.

Returns:

Stream of blurred vectors with same labels.

asMap

default Map<Vector,Double> asMap(int datasetSize)

Convert first N values from stream to map.

Parameters:

datasetSize - Dataset size.

Returns:

Map of vectors and labels.

asDatasetBuilder

default DatasetBuilder<Vector,Double> asDatasetBuilder(int datasetSize,
                                                       int partitions)

Convert first N values from stream to DatasetBuilder.

Parameters:

datasetSize - Dataset size.

partitions - Partitions count.

Returns:

Dataset builder.

asDatasetBuilder

default DatasetBuilder<Vector,Double> asDatasetBuilder(int datasetSize,
                                                       IgniteBiPredicate<Vector,Double> filter,
                                                       int partitions)

Convert first N values from stream to DatasetBuilder.

Parameters:

datasetSize - Dataset size.

filter - Data filter.

partitions - Partitions count.

Returns:

Dataset builder.

asDatasetBuilder

default DatasetBuilder<Vector,Double> asDatasetBuilder(int datasetSize,
                                                       IgniteBiPredicate<Vector,Double> filter,
                                                       int partitions,
                                                       UpstreamTransformerBuilder upstreamTransformerBuilder)

Convert first N values from stream to DatasetBuilder.

Parameters:

datasetSize - Dataset size.

filter - Data filter.

partitions - Partitions count.

upstreamTransformerBuilder - Upstream transformer builder.

Returns:

Dataset builder.

fillCacheWithCustomKey

default <K> void fillCacheWithCustomKey(int datasetSize,
                                        IgniteCache<K,LabeledVector<Double>> cache,
                                        Function<LabeledVector<Double>,K> keyMapper)

Fills given cache with labeled vectors from this generator and user defined mapper from vectors to keys.

Type Parameters:

K - Key type.

Parameters:

datasetSize - Rows count to put.

cache - Cache.

keyMapper - Mapping from vectors to keys.

fillCacheWithVecHashAsKey

default void fillCacheWithVecHashAsKey(int datasetSize,
                                       IgniteCache<Integer,LabeledVector<Double>> cache)

Fills given cache with labeled vectors from this generator as values and their hashcodes as keys.

Parameters:

datasetSize - Rows count to put.

cache - Cache.

fillCacheWithVecUUIDAsKey

default void fillCacheWithVecUUIDAsKey(int datasetSize,
                                       IgniteCache<UUID,LabeledVector<Double>> cache)

Fills given cache with labeled vectors from this generator as values and random UUIDs as keys

Parameters:

datasetSize - Rows count to put.

cache - Cache.