org.apache.ignite.spi.checkpoint.s3

Class S3CheckpointSpi

java.lang.Object

org.apache.ignite.spi.IgniteSpiAdapter

org.apache.ignite.spi.checkpoint.s3.S3CheckpointSpi

All Implemented Interfaces:

CheckpointSpi, IgniteSpi

@IgniteSpiMultipleInstancesSupport(value=true)
public class S3CheckpointSpi
extends IgniteSpiAdapter
implements CheckpointSpi

This class defines Amazon S3-based implementation for checkpoint SPI.

For information about Amazon S3 visit aws.amazon.com.

Configuration

Mandatory

This SPI has one mandatory configuration parameter:

• AWS credentials (see setAwsCredentials(AWSCredentials)

Optional

This SPI has following optional configuration parameters:

• Bucket name suffix (see setBucketNameSuffix(String))
• Client configuration (see setClientConfiguration(ClientConfiguration))
• Bucket endpoint (see setBucketEndpoint(String))
• Server side encryption algorithm (see setSSEAlgorithm(String))

Java Example

S3CheckpointSpi can be configured as follows:

 IgniteConfiguration cfg = new IgniteConfiguration();

 S3CheckpointSpi spi = new S3CheckpointSpi();

 AWSCredentials cred = new BasicAWSCredentials(YOUR_ACCESS_KEY_ID, YOUR_SECRET_ACCESS_KEY);

 spi.setAwsCredentials(cred);

 spi.setBucketNameSuffix("checkpoints");

 // Override default checkpoint SPI.
 cfg.setCheckpointSpi(cpSpi);

 // Start grid.
 G.start(cfg);
 

Spring Example

S3CheckpointSpi can be configured from Spring XML configuration file:

 <bean id="grid.custom.cfg" class="org.apache.ignite.configuration.IgniteConfiguration" singleton="true">
     ...
        <property name="checkpointSpi">
            <bean class="org.apache.ignite.spi.checkpoint.s3.S3CheckpointSpi">
                <property name="awsCredentials">
                    <bean class="com.amazonaws.auth.BasicAWSCredentials">
                        <constructor-arg value="YOUR_ACCESS_KEY_ID" />
                        <constructor-arg value="YOUR_SECRET_ACCESS_KEY" />
                    </bean>
                </property>
            </bean>
        </property>
     ...
 </bean>
 

Note that storing data in AWS S3 service will result in charges to your AWS account. Choose another implementation of CheckpointSpi for local or home network tests.

For information about Spring framework visit www.springframework.org

See Also:

CheckpointSpi

Field Summary

Fields

Modifier and Type	Field and Description
static String	BUCKET_NAME_PREFIX Prefix to use in bucket name generation.
static String	DFLT_BUCKET_NAME_SUFFIX Suffix to use in bucket name generation.

Fields inherited from class org.apache.ignite.spi.IgniteSpiAdapter

igniteInstanceName

Constructor Summary

Constructors

Constructor and Description
S3CheckpointSpi()

Method Summary

Modifier and Type	Method and Description
String	getAccessKey() Gets S3 access key.
@Nullable String	getBucketEndpoint() Gets S3 bucket endpoint to use.
String	getBucketName() Gets S3 bucket name to use.
String	getProxyHost() Gets HTTP proxy host.
String	getProxyPassword() Gets HTTP proxy password.
int	getProxyPort() Gets HTTP proxy port.
String	getProxyUsername() Gets HTTP proxy user name.
String	getSecretAccessKey() Gets S3 secret key.
@Nullable String	getSSEAlgorithm() Gets S3 server-side encryption algorithm.
byte[]	loadCheckpoint(String key) Loads checkpoint from storage by its unique key.
boolean	removeCheckpoint(String key) This method instructs the checkpoint provider to clean saved data for a given key.
boolean	saveCheckpoint(String key, byte[] state, long timeout, boolean overwrite) Saves checkpoint to the storage.
S3CheckpointSpi	setAwsCredentials(com.amazonaws.auth.AWSCredentials cred) Sets AWS credentials.
S3CheckpointSpi	setBucketEndpoint(String bucketEndpoint) Sets bucket endpoint.
S3CheckpointSpi	setBucketNameSuffix(String bucketNameSuffix) Sets bucket name suffix.
void	setCheckpointListener(CheckpointListener lsnr) Sets the checkpoint listener.
S3CheckpointSpi	setClientConfiguration(com.amazonaws.ClientConfiguration cfg) Sets Amazon client configuration.
S3CheckpointSpi	setName(String name) Sets SPI name.
S3CheckpointSpi	setSSEAlgorithm(String sseAlg) Sets server-side encryption algorithm for Amazon S3-managed encryption keys.
void	spiStart(String igniteInstanceName) This method is called to start SPI.
void	spiStop() This method is called to stop SPI.
String	toString()

Methods inherited from class org.apache.ignite.spi.IgniteSpiAdapter

addTimeoutObject, assertParameter, checkConfigurationConsistency0, clientFailureDetectionTimeout, configInfo, createSpiAttributeName, failureDetectionTimeout, failureDetectionTimeoutEnabled, failureDetectionTimeoutEnabled, getConsistentAttributeNames, getExceptionRegistry, getLocalNode, getName, getNodeAttributes, getSpiContext, ignite, initFailureDetectionTimeout, injectables, injectResources, isNodeStopping, onBeforeStart, onClientDisconnected, onClientReconnected, onContextDestroyed, onContextDestroyed0, onContextInitialized, onContextInitialized0, registerMBean, removeTimeoutObject, started, startInfo, startStopwatch, stopInfo, unregisterMBean

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.ignite.spi.IgniteSpi

getName, getNodeAttributes, onClientDisconnected, onClientReconnected, onContextDestroyed, onContextInitialized

Field Detail

BUCKET_NAME_PREFIX

public static final String BUCKET_NAME_PREFIX

Prefix to use in bucket name generation.

See Also:

Constant Field Values

DFLT_BUCKET_NAME_SUFFIX

public static final String DFLT_BUCKET_NAME_SUFFIX

Suffix to use in bucket name generation.

See Also:

Constant Field Values

Constructor Detail

S3CheckpointSpi

public S3CheckpointSpi()

Method Detail

getBucketName

public String getBucketName()

Gets S3 bucket name to use.

Returns:

S3 bucket name to use.

getBucketEndpoint

@Nullable
public @Nullable String getBucketEndpoint()

Gets S3 bucket endpoint to use.

Returns:

S3 bucket endpoint to use.

getSSEAlgorithm

@Nullable
public @Nullable String getSSEAlgorithm()

Gets S3 server-side encryption algorithm.

Returns:

S3 server-side encryption algorithm to use.

getAccessKey

public String getAccessKey()

Gets S3 access key.

Returns:

S3 access key.

getSecretAccessKey

public String getSecretAccessKey()

Gets S3 secret key.

Returns:

S3 secret key.

getProxyHost

public String getProxyHost()

Gets HTTP proxy host.

Returns:

HTTP proxy host.

getProxyPort

public int getProxyPort()

Gets HTTP proxy port.

Returns:

HTTP proxy port.

getProxyUsername

public String getProxyUsername()

Gets HTTP proxy user name.

Returns:

HTTP proxy user name.

getProxyPassword

public String getProxyPassword()

Gets HTTP proxy password.

Returns:

HTTP proxy password.

setBucketNameSuffix

@IgniteSpiConfiguration(optional=true)
public S3CheckpointSpi setBucketNameSuffix(String bucketNameSuffix)

Sets bucket name suffix.

Parameters:

bucketNameSuffix - Bucket name suffix.

Returns:

this for chaining.

setBucketEndpoint

@IgniteSpiConfiguration(optional=true)
public S3CheckpointSpi setBucketEndpoint(String bucketEndpoint)

Sets bucket endpoint. If the endpoint is not set then S3CheckpointSpi will go to each region to find a corresponding bucket. For information about possible endpoint names visit docs.aws.amazon.com

Parameters:

bucketEndpoint - Bucket endpoint, for example, s3.us-east-2.amazonaws.com.

Returns:

this for chaining.

setSSEAlgorithm

@IgniteSpiConfiguration(optional=true)
public S3CheckpointSpi setSSEAlgorithm(String sseAlg)

Sets server-side encryption algorithm for Amazon S3-managed encryption keys. For information about possible S3-managed encryption keys visit docs.aws.amazon.com.

Parameters:

sseAlg - Server-side encryption algorithm, for example, AES256 or SSES3.

Returns:

this for chaining.

setClientConfiguration

@IgniteSpiConfiguration(optional=true)
public S3CheckpointSpi setClientConfiguration(com.amazonaws.ClientConfiguration cfg)

Sets Amazon client configuration.

For details refer to Amazon S3 API reference.

Parameters:

cfg - Amazon client configuration.

Returns:

this for chaining.

setAwsCredentials

@IgniteSpiConfiguration(optional=false)
public S3CheckpointSpi setAwsCredentials(com.amazonaws.auth.AWSCredentials cred)

Sets AWS credentials.

For details refer to Amazon S3 API reference.

Parameters:

cred - AWS credentials.

Returns:

this for chaining.

spiStart

public void spiStart(String igniteInstanceName)
              throws IgniteSpiException

This method is called to start SPI. After this method returns successfully kernel assumes that SPI is fully operational.

Specified by:

spiStart in interface IgniteSpi

Parameters:

igniteInstanceName - Name of Ignite instance this SPI is being started for (null for default Ignite instance).

Throws:

IgniteSpiException - Throws in case of any error during SPI start.

spiStop

public void spiStop()
             throws IgniteSpiException

This method is called to stop SPI. After this method returns kernel assumes that this SPI is finished and all resources acquired by it are released.

Note that this method can be called at any point including during recovery of failed start. It should make no assumptions on what state SPI will be in when this method is called.

Specified by:

spiStop in interface IgniteSpi

Throws:

IgniteSpiException - Thrown in case of any error during SPI stop.

loadCheckpoint

public byte[] loadCheckpoint(String key)
                      throws IgniteSpiException

Loads checkpoint from storage by its unique key.

Specified by:

loadCheckpoint in interface CheckpointSpi

Parameters:

key - Checkpoint key.

Returns:

Loaded data or null if there is no data for a given key.

Throws:

IgniteSpiException - Thrown in case of any error while loading checkpoint data. Note that in case when given key is not found this method will return null.

saveCheckpoint

public boolean saveCheckpoint(String key,
                              byte[] state,
                              long timeout,
                              boolean overwrite)
                       throws IgniteSpiException

Saves checkpoint to the storage.

Specified by:

saveCheckpoint in interface CheckpointSpi

Parameters:

key - Checkpoint unique key.

state - Saved data.

timeout - Every intermediate data stored by checkpoint provider should have a timeout. Timeout allows for effective resource management by checkpoint provider by cleaning saved data that are not needed anymore. Generally, the user should choose the minimum possible timeout to avoid long-term resource acquisition by checkpoint provider. Value 0 means that timeout will never expire.

overwrite - Whether or not overwrite checkpoint if it already exists.

Returns:

true if checkpoint has been actually saved, false otherwise.

Throws:

IgniteSpiException - Thrown in case of any error while saving checkpoint data.

removeCheckpoint

public boolean removeCheckpoint(String key)

This method instructs the checkpoint provider to clean saved data for a given key.

Specified by:

removeCheckpoint in interface CheckpointSpi

Parameters:

key - Key for the checkpoint to remove.

Returns:

true if data has been actually removed, false otherwise.

setCheckpointListener

public void setCheckpointListener(CheckpointListener lsnr)

Sets the checkpoint listener.

Specified by:

setCheckpointListener in interface CheckpointSpi

Parameters:

lsnr - The listener to set or null.

setName

public S3CheckpointSpi setName(String name)

Sets SPI name.

Overrides:

setName in class IgniteSpiAdapter

Parameters:

name - SPI name.

Returns:

this for chaining.

toString

public String toString()

Overrides:

toString in class Object