org.apache.ignite.spi.deployment.uri

Class UriDeploymentSpi

java.lang.Object

org.apache.ignite.spi.IgniteSpiAdapter

org.apache.ignite.spi.deployment.uri.UriDeploymentSpi

All Implemented Interfaces:

DeploymentSpi, IgniteSpi

@IgniteSpiMultipleInstancesSupport(value=true)
 @IgniteSpiConsistencyChecked(optional=false)
public class UriDeploymentSpi
extends IgniteSpiAdapter
implements DeploymentSpi

Implementation of DeploymentSpi which can deploy tasks from different sources like file system folders, email and HTTP.

Description

There are different ways to deploy tasks in grid and every deploy method depends on selected source protocol. This SPI is configured to work with a list of URI's. Every URI contains all data about protocol/transport plus configuration parameters like credentials, scan frequency, and others.

When SPI establishes a connection with an URI, it downloads deployable units to the temporary directory in order to prevent it from any changes while scanning. Use method setTemporaryDirectoryPath(String)) to set custom temporary folder for downloaded deployment units. SPI will create folder under the path with name identical to local node ID.

SPI tracks all changes of every given URI. This means that if any file is changed or deleted, SPI will re-deploy or delete corresponding tasks. Note that the very first apply to findResource(String) is blocked until SPI finishes scanning all URI's at least once.

There are several deployable unit types supported:

• JAR file.
• GAR file.
• Local disk folder with structure of an unpacked deployment archive.
• Local disk folder containing only compiled Java classes.

Deployment package

Deployment package can be represented as a regular JAR file with a specific structure similar to WAR format. Package files can have '.jar' or '.gar' extension.

Package structure:

      META-INF/
              |
               - ignite.xml
               - ...
      lib/
         |
          -some-lib.jar
          - ...
      xyz.class
      ...

• META-INF/ entry may contain ignite.xml file which is a task descriptor file. The purpose of task descriptor XML file is to specify all tasks to be deployed. This file is a regular Spring XML definition file. META-INF/ entry may also contain any other file specified by JAR format.
• lib/ entry contains all library dependencies.
• Compiled Java classes must be placed in the root of a package file.

A package may be deployed without a descriptor file. If there is no descriptor file, SPI will scan all classes in archive and instantiate those that implement ComputeTask interface. In that case, all grid task classes must have a public no-argument constructor. Use ComputeTaskAdapter adapter for convenience when creating grid tasks.

By default, all downloaded packages that have digital signature in META-INF folder will be verified and deployed only if signature is valid.

URI

This SPI uses a hierarchical URI definition. For more information about standard URI syntax refer to java.net.URI documentation.

[scheme:][//authority][path][?query][#fragment]

Every URI defines its own deployment repository which will be scanned for any changes. URI itself has all information about protocol, connectivity, scan intervals and other parameters.

URI's may contain special characters, like spaces. If encodeUri flag is set to true (see setEncodeUri(boolean)), then URI 'path' field will be automatically encoded. By default this flag is set to true.

Code Example

The following example demonstrates how the deployment SPI can be used. It expects that you have a package file in 'home/username/ignite/work/my_deployment/file' folder which contains 'myproject.HelloWorldTask' class.

 IgniteConfiguration cfg = new IgniteConfiguration();

 DeploymentSpi deploymentSpi = new UriDeploymentSpi();

 deploymentSpi.setUriList(Arrays.asList("file:///home/username/ignite/work/my_deployment/file"));

 cfg.setDeploymentSpi(deploymentSpi);

 try (Ignite ignite = Ignition.start(cfg)) {
     ignite.compute().execute("myproject.HelloWorldTask", "my args");
 }
 

Configuration

UriDeploymentSpi has the following optional configuration parameters (there are no mandatory parameters):

• Array of UriDeploymentScanner-s which will be used to deploy resources (see setScanners(UriDeploymentScanner...)). If not specified, preconfigured UriDeploymentFileScanner and UriDeploymentHttpScanner are used. You can implement your own scanner by implementing UriDeploymentScanner interface.
• Temporary directory path where scanned packages are copied to (see setTemporaryDirectoryPath(String)).
• List of URIs to scan (see setUriList(List)). If not specified, then URI specified by DFLT_DEPLOY_DIR is used.
• Flag to control encoding of the 'path' portion of URI (see setEncodeUri(boolean)).

Protocols

Following protocols are supported by this SPI out of the box:

• file:// - File protocol
• http:// - HTTP protocol
• https:// - Secure HTTP protocol

Custom Protocols.

You can add support for additional protocols if needed. To do this implement UriDeploymentScanner interface and plug your implementation into the SPI via setScanners(UriDeploymentScanner...) method.

In addition to SPI configuration parameters, all necessary configuration parameters for selected URI should be defined in URI. Different protocols have different configuration parameters described below. Parameters are separated by ';' character.

File

For this protocol SPI will scan folder specified by URI on file system and download any deplloyment files or directories that end with .jar or .gar from source directory defined in URI. For file system URI must have scheme equal to file.

Following parameters are supported:

Parameter	Description	Optional	Default
freq	Scanning frequency in milliseconds.	Yes	5000 ms specified in UriDeploymentFileScanner.DFLT_SCAN_FREQ.

File URI Example

The following example will scan 'c:/Program files/ignite/deployment' folder on local box every '1000' milliseconds. Note that since path has spaces, setEncodeUri(boolean) parameter must be set to true (which is default behavior).

file://freq=2000@localhost/c:/Program files/ignite/deployment

HTTP/HTTPS

URI deployment scanner tries to read DOM of the html it points to and parses out href attributes of all <a> tags - this becomes the collection of URLs to deployment package files that should be deployed. It's important that HTTP scanner uses URLConnection.getLastModified() method to check if there were any changes since last iteration for each deployment file before redeploying.

Following parameters are supported:

Parameter	Description	Optional	Default
freq	Scanning frequency in milliseconds.	Yes	300000 ms specified in UriDeploymentHttpScanner.DFLT_SCAN_FREQ.

HTTP URI Example

The following example will download the page `www.mysite.com/ignite/deployment`, parse it and download and deploy all deployment packages specified by href attributes of <a> elements on the page using authentication 'username:password' every '10000' milliseconds (only new/updated packages).

http://username:password;freq=10000@www.mysite.com:110/ignite/deployment

Java Example

UriDeploymentSpi needs to be explicitly configured to override default local deployment SPI.

 UriDeploymentSpi deploySpi = new UriDeploymentSpi();

 IgniteConfiguration cfg = new IgniteConfiguration();

 // Set URIs.
 deploySpi.setUriList(Arrays.asList("http://www.site.com/tasks",
     "file://freq=20000@localhost/c:/Program files/gg-deployment"));

 // Override temporary directory path.
 deploySpi.setTemporaryDirectoryPath("c:/tmp/grid");

 //  Override default deployment SPI.
 cfg.setDeploymentSpi(deploySpi);

 //  Start grid.
 Ignition.start(cfg);
 

Spring Example

UriDeploymentSpi can be configured from Spring XML configuration file:

 <bean id="grid.custom.cfg" class="org.apache.ignite.configuration.IgniteConfiguration" singleton="true">
         ...
         <property name="deploymentSpi">
             <bean class="org.apache.ignite.grid.spi.deployment.uri.UriDeploymentSpi">
                 <property name="temporaryDirectoryPath" value="c:/tmp/grid"/>
                 <property name="uriList">
                     <list>
                         <value>http://www.site.com/tasks</value>
                         <value>file://freq=20000@localhost/c:/Program files/gg-deployment</value>
                     </list>
                 </property>
             </bean>
         </property>
         ...
 </bean>
 

For information about Spring framework visit www.springframework.org

See Also:

DeploymentSpi

Field Summary

Fields

Modifier and Type	Field and Description
static String	DEPLOY_TMP_ROOT_NAME Default temporary directory name relative to file path setTemporaryDirectoryPath(String)} (value is gg.uri.deployment.tmp).
static String	DFLT_DEPLOY_DIR Default deployment directory where SPI will pick up packages.
static String	XML_DESCRIPTOR_PATH Default task description file path and name (value is META-INF/ignite.xml).

Fields inherited from class org.apache.ignite.spi.IgniteSpiAdapter

ignite, igniteInstanceName

Constructor Summary

Constructors

Constructor and Description
UriDeploymentSpi()

Method Summary

Modifier and Type	Method and Description
@Nullable DeploymentResource	findResource(String rsrcName) Finds class loader for the given class.
org.apache.ignite.spi.deployment.uri.scanners.UriDeploymentScanner[]	getScanners() Gets scanners.
String	getTemporaryDirectoryPath() Gets temporary directory path.
List<String>	getUriList() Gets list of URIs that are processed by SPI.
boolean	isCheckMd5() Gets checkMd5 property.
boolean	register(ClassLoader ldr, Class<?> rsrc) Registers a class loader with this SPI.
UriDeploymentSpi	setCheckMd5(boolean checkMd5) If set to true then SPI should exclude files with same md5s from deployment.
UriDeploymentSpi	setEncodeUri(boolean encodeUri) Indicates that URI must be encoded before usage.
void	setListener(@Nullable DeploymentListener lsnr) Sets or unsets deployment event listener.
IgniteSpiAdapter	setName(String name) Sets SPI name.
UriDeploymentSpi	setScanners(org.apache.ignite.spi.deployment.uri.scanners.UriDeploymentScanner... scanners) Sets scanners.
UriDeploymentSpi	setTemporaryDirectoryPath(String tmpDirPath) Sets absolute path to temporary directory which will be used by deployment SPI to keep all deployed classes in.
UriDeploymentSpi	setUriList(List<String> uriList) Sets list of URI which point to a directory with packages and which should be scanned by SPI for the new tasks.
void	spiStart(String igniteInstanceName) This method is called to start SPI.
void	spiStop() This method is called to stop SPI.
String	toString()
boolean	unregister(String rsrcName) Unregisters all class loaders that have a class with given name or have a class with give ComputeTaskName value.

Methods inherited from class org.apache.ignite.spi.IgniteSpiAdapter

addTimeoutObject, assertParameter, checkConfigurationConsistency0, clientFailureDetectionTimeout, configInfo, createSpiAttributeName, failureDetectionTimeout, failureDetectionTimeoutEnabled, failureDetectionTimeoutEnabled, getConsistentAttributeNames, getExceptionRegistry, getLocalNode, getName, getNodeAttributes, getSpiContext, ignite, initFailureDetectionTimeout, injectables, injectResources, isNodeStopping, onBeforeStart, onClientDisconnected, onClientReconnected, onContextDestroyed, onContextDestroyed0, onContextInitialized, onContextInitialized0, registerMBean, removeTimeoutObject, started, startInfo, startStopwatch, stopInfo, unregisterMBean

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.ignite.spi.IgniteSpi

getName, getNodeAttributes, onClientDisconnected, onClientReconnected, onContextDestroyed, onContextInitialized

Field Detail

DFLT_DEPLOY_DIR

public static final String DFLT_DEPLOY_DIR

Default deployment directory where SPI will pick up packages. Note that this path is relative to IGNITE_HOME/work folder if IGNITE_HOME system or environment variable specified, otherwise it is relative to work folder under system java.io.tmpdir folder.

See Also:

IgniteConfiguration.getWorkDirectory(), Constant Field Values

XML_DESCRIPTOR_PATH

public static final String XML_DESCRIPTOR_PATH

Default task description file path and name (value is META-INF/ignite.xml).

See Also:

Constant Field Values

DEPLOY_TMP_ROOT_NAME

public static final String DEPLOY_TMP_ROOT_NAME

Default temporary directory name relative to file path setTemporaryDirectoryPath(String)} (value is gg.uri.deployment.tmp).

See Also:

Constant Field Values

Constructor Detail

UriDeploymentSpi

public UriDeploymentSpi()

Method Detail

setTemporaryDirectoryPath

@IgniteSpiConfiguration(optional=true)
public UriDeploymentSpi setTemporaryDirectoryPath(String tmpDirPath)

Sets absolute path to temporary directory which will be used by deployment SPI to keep all deployed classes in.

If not provided, default value is java.io.tmpdir system property value.

Parameters:

tmpDirPath - Temporary directory path.

Returns:

this for chaining.

setUriList

@IgniteSpiConfiguration(optional=true)
public UriDeploymentSpi setUriList(List<String> uriList)

Sets list of URI which point to a directory with packages and which should be scanned by SPI for the new tasks.

If not provided, default value is list with file://${IGNITE_HOME}/work/deployment/file element. Note that system property IGNITE_HOME must be set. For unknown IGNITE_HOME list of URI must be provided explicitly.

Parameters:

uriList - Package file URIs.

Returns:

this for chaining.

setCheckMd5

@IgniteSpiConfiguration(optional=true)
public UriDeploymentSpi setCheckMd5(boolean checkMd5)

If set to true then SPI should exclude files with same md5s from deployment. Otherwise it should try to load new unit regardless to possible file duplication.

Parameters:

checkMd5 - new value for the property

Returns:

this for chaining.

isCheckMd5

public boolean isCheckMd5()

Gets checkMd5 property.

Returns:

value of the checkMd5 property.

setEncodeUri

@IgniteSpiConfiguration(optional=true)
public UriDeploymentSpi setEncodeUri(boolean encodeUri)

Indicates that URI must be encoded before usage. Encoding means replacing all occurrences of space with '%20', percent sign with '%25' and semicolon with '%3B'.

If not provided, default value is true.

Parameters:

encodeUri - true if every URI should be encoded and false otherwise.

Returns:

this for chaining.

getTemporaryDirectoryPath

public String getTemporaryDirectoryPath()

Gets temporary directory path.

Returns:

Temporary directory path.

getUriList

public List<String> getUriList()

Gets list of URIs that are processed by SPI.

Returns:

List of URIs.

setListener

public void setListener(@Nullable
                        @Nullable DeploymentListener lsnr)

Sets or unsets deployment event listener. Grid implementation will use this listener to properly add and remove various deployments.

Specified by:

setListener in interface DeploymentSpi

Parameters:

lsnr - Listener for deployment events. null to unset the listener.

getScanners

public org.apache.ignite.spi.deployment.uri.scanners.UriDeploymentScanner[] getScanners()

Gets scanners.

Returns:

Scanners.

setScanners

@IgniteSpiConfiguration(optional=true)
public UriDeploymentSpi setScanners(org.apache.ignite.spi.deployment.uri.scanners.UriDeploymentScanner... scanners)

Sets scanners.

Parameters:

scanners - Scanners.

Returns:

this for chaining.

spiStop

public void spiStop()
             throws IgniteSpiException

This method is called to stop SPI. After this method returns kernel assumes that this SPI is finished and all resources acquired by it are released.

Note that this method can be called at any point including during recovery of failed start. It should make no assumptions on what state SPI will be in when this method is called.

Specified by:

spiStop in interface IgniteSpi

Throws:

IgniteSpiException - Thrown in case of any error during SPI stop.

spiStart

public void spiStart(String igniteInstanceName)
              throws IgniteSpiException

This method is called to start SPI. After this method returns successfully kernel assumes that SPI is fully operational.

Specified by:

spiStart in interface IgniteSpi

Parameters:

igniteInstanceName - Name of Ignite instance this SPI is being started for (null for default Ignite instance).

Throws:

IgniteSpiException - Throws in case of any error during SPI start.

findResource

@Nullable
public @Nullable DeploymentResource findResource(String rsrcName)

Finds class loader for the given class.

Specified by:

findResource in interface DeploymentSpi

Parameters:

rsrcName - Class name or class alias to find class loader for.

Returns:

Deployed class loader, or null if not deployed.

register

public boolean register(ClassLoader ldr,
                        Class<?> rsrc)
                 throws IgniteSpiException

Registers a class loader with this SPI. This method exists to be able to add external class loaders to deployment SPI. Deployment SPI may also have its own class loaders. For example, in case of GAR deployment, every GAR file is loaded and deployed with a separate class loader maintained internally by the SPI.

The array of classes passed in should be checked for presence of ComputeTaskName annotations. The classes that have this annotation should be accessible by this name from DeploymentSpi.findResource(String) method.

Specified by:

register in interface DeploymentSpi

Parameters:

ldr - Class loader to register.

rsrc - Class that should be checked for aliases. Currently the only alias in the system is ComputeTaskName for task classes; in future, there may be others.

Returns:

True if resource was registered.

Throws:

IgniteSpiException - If registration failed.

unregister

public boolean unregister(String rsrcName)

Unregisters all class loaders that have a class with given name or have a class with give ComputeTaskName value.

Specified by:

unregister in interface DeploymentSpi

Parameters:

rsrcName - Either class name or ComputeTaskName value for a class whose class loader needs to be unregistered.

Returns:

True if resource was unregistered.

setName

public IgniteSpiAdapter setName(String name)

Sets SPI name.

Overrides:

setName in class IgniteSpiAdapter

Parameters:

name - SPI name.

Returns:

this for chaining.

toString

public String toString()

Overrides:

toString in class Object