K
- Type of a key in upstream
data.V
- Type of a value in upstream
data.public class CacheBasedDatasetBuilder<K,V> extends Object implements DatasetBuilder<K,V>
CacheBasedDataset
. Encapsulate logic of building cache based dataset such as
allocation required data structures and initialization of context
part of partitions.Modifier and Type | Field and Description |
---|---|
static int |
DEFAULT_NUMBER_OF_RETRIES
Default number of retries for the case when one of partitions not found on the node where loading is performed.
|
Constructor and Description |
---|
CacheBasedDatasetBuilder(Ignite ignite,
IgniteCache<K,V> upstreamCache)
Constructs a new instance of cache based dataset builder that makes
CacheBasedDataset with default
predicate that passes all upstream entries to dataset. |
CacheBasedDatasetBuilder(Ignite ignite,
IgniteCache<K,V> upstreamCache,
IgniteBiPredicate<K,V> filter)
Constructs a new instance of cache based dataset builder that makes
CacheBasedDataset . |
CacheBasedDatasetBuilder(Ignite ignite,
IgniteCache<K,V> upstreamCache,
IgniteBiPredicate<K,V> filter,
UpstreamTransformerBuilder transformerBuilder)
Constructs a new instance of cache based dataset builder that makes
CacheBasedDataset . |
CacheBasedDatasetBuilder(Ignite ignite,
IgniteCache<K,V> upstreamCache,
IgniteBiPredicate<K,V> filter,
UpstreamTransformerBuilder transformerBuilder,
Boolean isKeepBinary,
int retries)
Constructs a new instance of cache based dataset builder that makes
CacheBasedDataset . |
Modifier and Type | Method and Description |
---|---|
<C extends Serializable,D extends AutoCloseable> |
build(LearningEnvironmentBuilder envBuilder,
PartitionContextBuilder<K,V,C> partCtxBuilder,
PartitionDataBuilder<K,V,C,D> partDataBuilder,
LearningEnvironment localLearningEnv)
Constructs a new instance of
Dataset that includes allocation required data structures and
initialization of context part of partitions. |
DatasetBuilder<K,V> |
withFilter(IgniteBiPredicate<K,V> filterToAdd)
Returns new instance of DatasetBuilder using conjunction of internal filter and
filterToAdd . |
CacheBasedDatasetBuilder<K,V> |
withKeepBinary(boolean isKeepBinary)
Add keepBinary policy.
|
CacheBasedDatasetBuilder<K,V> |
withRetriesNumber(int retries)
Sets number of retries. 15 * 60 by default.
|
DatasetBuilder<K,V> |
withUpstreamTransformer(UpstreamTransformerBuilder builder)
Returns new instance of
DatasetBuilder with new UpstreamTransformerBuilder added
to chain of upstream transformer builders. |
public static final int DEFAULT_NUMBER_OF_RETRIES
public CacheBasedDatasetBuilder(Ignite ignite, IgniteCache<K,V> upstreamCache)
CacheBasedDataset
with default
predicate that passes all upstream entries to dataset.ignite
- Ignite instance.upstreamCache
- Ignite Cache with upstream
data.public CacheBasedDatasetBuilder(Ignite ignite, IgniteCache<K,V> upstreamCache, IgniteBiPredicate<K,V> filter)
CacheBasedDataset
.ignite
- Ignite instance.upstreamCache
- Ignite Cache with upstream
data.filter
- Filter for upstream
data.public CacheBasedDatasetBuilder(Ignite ignite, IgniteCache<K,V> upstreamCache, IgniteBiPredicate<K,V> filter, UpstreamTransformerBuilder transformerBuilder)
CacheBasedDataset
.ignite
- Ignite instance.upstreamCache
- Ignite Cache with upstream
data.filter
- Filter for upstream
data.public CacheBasedDatasetBuilder(Ignite ignite, IgniteCache<K,V> upstreamCache, IgniteBiPredicate<K,V> filter, UpstreamTransformerBuilder transformerBuilder, Boolean isKeepBinary, int retries)
CacheBasedDataset
.ignite
- Ignite.upstreamCache
- Upstream cache.filter
- Filter.transformerBuilder
- Transformer builder.isKeepBinary
- Is keep binary for upstream cache.retries
- Number of retries for the case when one of partitions not found on the node where loading is performed.public <C extends Serializable,D extends AutoCloseable> CacheBasedDataset<K,V,C,D> build(LearningEnvironmentBuilder envBuilder, PartitionContextBuilder<K,V,C> partCtxBuilder, PartitionDataBuilder<K,V,C,D> partDataBuilder, LearningEnvironment localLearningEnv)
Dataset
that includes allocation required data structures and
initialization of context
part of partitions.build
in interface DatasetBuilder<K,V>
C
- Type of a partition context
.D
- Type of a partition data
.envBuilder
- Learning environment builder.partCtxBuilder
- Partition context
builder.partDataBuilder
- Partition data
builder.localLearningEnv
- Local learning environment.public DatasetBuilder<K,V> withUpstreamTransformer(UpstreamTransformerBuilder builder)
DatasetBuilder
with new UpstreamTransformerBuilder
added
to chain of upstream transformer builders. When needed, each builder in chain first transformed into
UpstreamTransformer
, those are in turn composed together one after another forming
final UpstreamTransformer
.
This transformer is applied to upstream data before it is passed
to PartitionDataBuilder
and PartitionContextBuilder
. This is needed to allow
transformation to upstream data which are agnostic of any changes that happen after.
Such transformations may be used for deriving meta-algorithms such as bagging
(see BaggingUpstreamTransformer
).withUpstreamTransformer
in interface DatasetBuilder<K,V>
DatasetBuilder
with new UpstreamTransformerBuilder
added
to chain of upstream transformer builders.public DatasetBuilder<K,V> withFilter(IgniteBiPredicate<K,V> filterToAdd)
filterToAdd
.withFilter
in interface DatasetBuilder<K,V>
filterToAdd
- Additional filter.public CacheBasedDatasetBuilder<K,V> withKeepBinary(boolean isKeepBinary)
isKeepBinary
- Is keep binary.public CacheBasedDatasetBuilder<K,V> withRetriesNumber(int retries)
retries
- Number of retries.
GridGain In-Memory Computing Platform : ver. 8.9.14 Release Date : November 5 2024