UriDeploymentSpi (Ignite 2.7.2)

java.lang.Object
- org.apache.ignite.spi.IgniteSpiAdapter
- - org.apache.ignite.spi.deployment.uri.UriDeploymentSpi

All Implemented Interfaces:: DeploymentSpi, IgniteSpi

@IgniteSpiMultipleInstancesSupport(value=true)
 @IgniteSpiConsistencyChecked(optional=false)
public class UriDeploymentSpi
extends IgniteSpiAdapter
implements DeploymentSpi

Implementation of DeploymentSpi which can deploy tasks from different sources like file system folders, email and HTTP. There are different ways to deploy tasks in grid and every deploy method depends on selected source protocol. This SPI is configured to work with a list of URI's. Every URI contains all data about protocol/transport plus configuration parameters like credentials, scan frequency, and others.

When SPI establishes a connection with an URI, it downloads deployable units to the temporary directory in order to prevent it from any changes while scanning. Use method setTemporaryDirectoryPath(String)) to set custom temporary folder for downloaded deployment units. SPI will create folder under the path with name identical to local node ID.

SPI tracks all changes of every given URI. This means that if any file is changed or deleted, SPI will re-deploy or delete corresponding tasks. Note that the very first apply to findResource(String) is blocked until SPI finishes scanning all URI's at least once.

There are several deployable unit types supported:

GAR file.
Local disk folder with structure of unpacked GAR file.
Local disk folder containing only compiled Java classes.

GAR file is a deployable unit. GAR file is based on ZLIB compression format like simple JAR file and its structure is similar to WAR archive. GAR file has '.gar' extension.

GAR file structure (file or directory ending with '.gar'):

      META-INF/
              |
               - ignite.xml
               - ...
      lib/
         |
          -some-lib.jar
          - ...
      xyz.class
      ...

META-INF/ entry may contain ignite.xml file which is a task descriptor file. The purpose of task descriptor XML file is to specify all tasks to be deployed. This file is a regular Spring XML definition file. META-INF/ entry may also contain any other file specified by JAR format.
lib/ entry contains all library dependencies.
Compiled Java classes must be placed in the root of a GAR file.

GAR file may be deployed without descriptor file. If there is no descriptor file, SPI will scan all classes in archive and instantiate those that implement ComputeTask interface. In that case, all grid task classes must have a public no-argument constructor. Use ComputeTaskAdapter adapter for convenience when creating grid tasks.

By default, all downloaded GAR files that have digital signature in META-INF folder will be verified and deployed only if signature is valid.

This SPI uses a hierarchical URI definition. For more information about standard URI syntax refer to java.net.URI documentation.

[scheme:][//authority][path][?query][#fragment]

Every URI defines its own deployment repository which will be scanned for any changes. URI itself has all information about protocol, connectivity, scan intervals and other parameters.

URI's may contain special characters, like spaces. If encodeUri flag is set to true (see setEncodeUri(boolean)), then URI 'path' field will be automatically encoded. By default this flag is set to true.

The following example demonstrates how the deployment SPI can be used. It expects that you have a GAR file in 'home/username/ignite/work/my_deployment/file' folder which contains 'myproject.HelloWorldTask' class.

 IgniteConfiguration cfg = new IgniteConfiguration();

 DeploymentSpi deploymentSpi = new UriDeploymentSpi();

 deploymentSpi.setUriList(Arrays.asList("file:///home/username/ignite/work/my_deployment/file"));

 cfg.setDeploymentSpi(deploymentSpi);

 try (Ignite ignite = Ignition.start(cfg)) {
     ignite.compute().execute("myproject.HelloWorldTask", "my args");
 }

UriDeploymentSpi has the following optional configuration parameters (there are no mandatory parameters):

Array of UriDeploymentScanner-s which will be used to deploy resources (see setScanners(UriDeploymentScanner...)). If not specified, preconfigured UriDeploymentFileScanner and UriDeploymentHttpScanner are used. You can implement your own scanner by implementing UriDeploymentScanner interface.
Temporary directory path where scanned GAR files and directories are copied to (see setTemporaryDirectoryPath(String)).
List of URIs to scan (see setUriList(List)). If not specified, then URI specified by DFLT_DEPLOY_DIR is used.
Flag to control encoding of the 'path' portion of URI (see setEncodeUri(boolean)).

Following protocols are supported by this SPI out of the box:

file:// - File protocol
http:// - HTTP protocol
https:// - Secure HTTP protocol

Custom Protocols.

You can add support for additional protocols if needed. To do this implement UriDeploymentScanner interface and plug your implementation into the SPI via setScanners(UriDeploymentScanner...) method.

In addition to SPI configuration parameters, all necessary configuration parameters for selected URI should be defined in URI. Different protocols have different configuration parameters described below. Parameters are separated by ';' character.

For this protocol SPI will scan folder specified by URI on file system and download any GAR files or directories that end with .gar from source directory defined in URI. For file system URI must have scheme equal to file.

Following parameters are supported:

Parameter	Description	Optional	Default
freq	Scanning frequency in milliseconds.	Yes	`5000` ms specified in `UriDeploymentFileScanner.DFLT_SCAN_FREQ`.

The following example will scan 'c:/Program files/ignite/deployment' folder on local box every '1000' milliseconds. Note that since path has spaces, setEncodeUri(boolean) parameter must be set to true (which is default behavior).

file://freq=2000@localhost/c:/Program files/ignite/deployment

URI deployment scanner tries to read DOM of the html it points to and parses out href attributes of all <a> tags - this becomes the collection of URLs to GAR files that should be deployed. It's important that HTTP scanner uses URLConnection.getLastModified() method to check if there were any changes since last iteration for each GAR-file before redeploying.

Following parameters are supported:

Parameter	Description	Optional	Default
freq	Scanning frequency in milliseconds.	Yes	`300000` ms specified in `UriDeploymentHttpScanner.DFLT_SCAN_FREQ`.

The following example will download the page `www.mysite.com/ignite/deployment`, parse it and download and deploy all GAR files specified by href attributes of <a> elements on the page using authentication 'username:password' every '10000' milliseconds (only new/updated GAR-s).

http://username:password;freq=10000@www.mysite.com:110/ignite/deployment

UriDeploymentSpi needs to be explicitly configured to override default local deployment SPI.

 UriDeploymentSpi deploySpi = new UriDeploymentSpi();

 IgniteConfiguration cfg = new IgniteConfiguration();

 // Set URIs.
 deploySpi.setUriList(Arrays.asList("http://www.site.com/tasks",
     "file://freq=20000@localhost/c:/Program files/gg-deployment"));

 // Override temporary directory path.
 deploySpi.setTemporaryDirectoryPath("c:/tmp/grid");

 //  Override default deployment SPI.
 cfg.setDeploymentSpi(deploySpi);

 //  Start grid.
 Ignition.start(cfg);

UriDeploymentSpi can be configured from Spring XML configuration file:

 <bean id="grid.custom.cfg" class="org.apache.ignite.configuration.IgniteConfiguration" singleton="true">
         ...
         <property name="deploymentSpi">
             <bean class="org.apache.ignite.grid.spi.deployment.uri.UriDeploymentSpi">
                 <property name="temporaryDirectoryPath" value="c:/tmp/grid"/>
                 <property name="uriList">
                     <list>
                         <value>http://www.site.com/tasks</value>
                         <value>file://freq=20000@localhost/c:/Program files/gg-deployment</value>
                     </list>
                 </property>
             </bean>
         </property>
         ...
 </bean>

For information about Spring framework visit www.springframework.org

See Also:: DeploymentSpi

Field Summary

Fields
Modifier and Type	Field and Description
`static String`	`DEPLOY_TMP_ROOT_NAME` Default temporary directory name relative to file path `setTemporaryDirectoryPath(String)`} (value is `gg.uri.deployment.tmp`).
`static String`	`DFLT_DEPLOY_DIR` Default deployment directory where SPI will pick up GAR files.
`static String`	`XML_DESCRIPTOR_PATH` Default task description file path and name (value is `META-INF/ignite.xml`).

Fields inherited from class org.apache.ignite.spi.IgniteSpiAdapter
ignite, igniteInstanceName

Constructor Summary

Constructors
Constructor and Description

UriDeploymentSpi()

Constructors
Constructor and Description
`UriDeploymentSpi()`

Method Summary

All Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`@Nullable DeploymentResource`	`findResource(String rsrcName)` Finds class loader for the given class.
`org.apache.ignite.spi.deployment.uri.scanners.UriDeploymentScanner[]`	`getScanners()` Gets scanners.
`String`	`getTemporaryDirectoryPath()` Gets temporary directory path.
`List<String>`	`getUriList()` Gets list of URIs that are processed by SPI.
`boolean`	`isCheckMd5()` Gets `checkMd5` property.
`boolean`	`register(ClassLoader ldr, Class<?> rsrc)` Registers a class loader with this SPI.
`UriDeploymentSpi`	`setCheckMd5(boolean checkMd5)` If set to `true` then SPI should exclude files with same md5s from deployment.
`UriDeploymentSpi`	`setEncodeUri(boolean encodeUri)` Indicates that URI must be encoded before usage.
`void`	`setListener(@Nullable DeploymentListener lsnr)` Sets or unsets deployment event listener.
`IgniteSpiAdapter`	`setName(String name)` Sets SPI name.
`UriDeploymentSpi`	`setScanners(org.apache.ignite.spi.deployment.uri.scanners.UriDeploymentScanner... scanners)` Sets scanners.
`UriDeploymentSpi`	`setTemporaryDirectoryPath(String tmpDirPath)` Sets absolute path to temporary directory which will be used by deployment SPI to keep all deployed classes in.
`UriDeploymentSpi`	`setUriList(List<String> uriList)` Sets list of URI which point to GAR file and which should be scanned by SPI for the new tasks.
`void`	`spiStart(String igniteInstanceName)` This method is called to start SPI.
`void`	`spiStop()` This method is called to stop SPI.
`String`	`toString()`
`boolean`	`unregister(String rsrcName)` Unregisters all class loaders that have a class with given name or have a class with give `ComputeTaskName` value.

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.ignite.spi.IgniteSpi
getName, getNodeAttributes, onClientDisconnected, onClientReconnected, onContextDestroyed, onContextInitialized

- Field Detail
  - DFLT_DEPLOY_DIR
```
public static final String DFLT_DEPLOY_DIR
```
    Default deployment directory where SPI will pick up GAR files. Note that this path is relative to IGNITE_HOME/work folder if IGNITE_HOME system or environment variable specified, otherwise it is relative to work folder under system java.io.tmpdir folder.
    
    See Also:
    
    IgniteConfiguration.getWorkDirectory(), Constant Field Values
  - XML_DESCRIPTOR_PATH
```
public static final String XML_DESCRIPTOR_PATH
```
    Default task description file path and name (value is META-INF/ignite.xml).
    
    See Also:
    
    Constant Field Values
  - DEPLOY_TMP_ROOT_NAME
```
public static final String DEPLOY_TMP_ROOT_NAME
```
    Default temporary directory name relative to file path setTemporaryDirectoryPath(String)} (value is gg.uri.deployment.tmp).
    
    See Also:
    
    Constant Field Values
- Constructor Detail
  - UriDeploymentSpi
```
public UriDeploymentSpi()
```
- Method Detail
  - setTemporaryDirectoryPath
```
@IgniteSpiConfiguration(optional=true)
public UriDeploymentSpi setTemporaryDirectoryPath(String tmpDirPath)
```
    Sets absolute path to temporary directory which will be used by deployment SPI to keep all deployed classes in.
    If not provided, default value is java.io.tmpdir system property value.
    
    Parameters:
    
    tmpDirPath - Temporary directory path.
    
    Returns:
    
    this for chaining.
  - setUriList
```
@IgniteSpiConfiguration(optional=true)
public UriDeploymentSpi setUriList(List<String> uriList)
```
    Sets list of URI which point to GAR file and which should be scanned by SPI for the new tasks.
    If not provided, default value is list with file://${IGNITE_HOME}/work/deployment/file element. Note that system property IGNITE_HOME must be set. For unknown IGNITE_HOME list of URI must be provided explicitly.
    
    Parameters:
    
    uriList - GAR file URIs.
    
    Returns:
    
    this for chaining.
  - setCheckMd5
```
@IgniteSpiConfiguration(optional=true)
public UriDeploymentSpi setCheckMd5(boolean checkMd5)
```
    If set to true then SPI should exclude files with same md5s from deployment. Otherwise it should try to load new unit regardless to possible file duplication.
    
    Parameters:
    
    checkMd5 - new value for the property
    
    Returns:
    
    this for chaining.
  - isCheckMd5
```
public boolean isCheckMd5()
```
    Gets checkMd5 property.
    
    Returns:
    
    value of the checkMd5 property.
  - setEncodeUri
```
@IgniteSpiConfiguration(optional=true)
public UriDeploymentSpi setEncodeUri(boolean encodeUri)
```
    Indicates that URI must be encoded before usage. Encoding means replacing all occurrences of space with '%20', percent sign with '%25' and semicolon with '%3B'.
    If not provided, default value is true.
    
    Parameters:
    
    encodeUri - true if every URI should be encoded and false otherwise.
    
    Returns:
    
    this for chaining.
  - getTemporaryDirectoryPath
```
public String getTemporaryDirectoryPath()
```
    Gets temporary directory path.
    
    Returns:
    
    Temporary directory path.
  - getUriList
```
public List<String> getUriList()
```
    Gets list of URIs that are processed by SPI.
    
    Returns:
    
    List of URIs.
  - setListener
```
public void setListener(@Nullable
                        @Nullable DeploymentListener lsnr)
```
    Sets or unsets deployment event listener. Grid implementation will use this listener to properly add and remove various deployments.
    
    Specified by:
    
    setListener in interface DeploymentSpi
    
    Parameters:
    
    lsnr - Listener for deployment events. null to unset the listener.
  - getScanners
```
public org.apache.ignite.spi.deployment.uri.scanners.UriDeploymentScanner[] getScanners()
```
    Gets scanners.
    
    Returns:
    
    Scanners.
  - setScanners
```
@IgniteSpiConfiguration(optional=true)
public UriDeploymentSpi setScanners(org.apache.ignite.spi.deployment.uri.scanners.UriDeploymentScanner... scanners)
```
    Sets scanners.
    
    Parameters:
    
    scanners - Scanners.
    
    Returns:
    
    this for chaining.
  - spiStop
```
public void spiStop()
             throws IgniteSpiException
```
    This method is called to stop SPI. After this method returns kernel assumes that this SPI is finished and all resources acquired by it are released.
    Note that this method can be called at any point including during recovery of failed start. It should make no assumptions on what state SPI will be in when this method is called.
    
    Specified by:
    
    spiStop in interface IgniteSpi
    
    Throws:
    
    IgniteSpiException - Thrown in case of any error during SPI stop.
  - spiStart
```
public void spiStart(String igniteInstanceName)
              throws IgniteSpiException
```
    This method is called to start SPI. After this method returns successfully kernel assumes that SPI is fully operational.
    
    Specified by:
    
    spiStart in interface IgniteSpi
    
    Parameters:
    
    igniteInstanceName - Name of Ignite instance this SPI is being started for (null for default Ignite instance).
    
    Throws:
    
    IgniteSpiException - Throws in case of any error during SPI start.
  - findResource
```
@Nullable
public @Nullable DeploymentResource findResource(String rsrcName)
```
    Finds class loader for the given class.
    
    Specified by:
    
    findResource in interface DeploymentSpi
    
    Parameters:
    
    rsrcName - Class name or class alias to find class loader for.
    
    Returns:
    
    Deployed class loader, or null if not deployed.
  - register
```
public boolean register(ClassLoader ldr,
                        Class<?> rsrc)
                 throws IgniteSpiException
```
    Registers a class loader with this SPI. This method exists to be able to add external class loaders to deployment SPI. Deployment SPI may also have its own class loaders. For example, in case of GAR deployment, every GAR file is loaded and deployed with a separate class loader maintained internally by the SPI.
    The array of classes passed in should be checked for presence of ComputeTaskName annotations. The classes that have this annotation should be accessible by this name from DeploymentSpi.findResource(String) method.
    
    Specified by:
    
    register in interface DeploymentSpi
    
    Parameters:
    
    ldr - Class loader to register.
    
    rsrc - Class that should be checked for aliases. Currently the only alias in the system is ComputeTaskName for task classes; in future, there may be others.
    
    Returns:
    
    True if resource was registered.
    
    Throws:
    
    IgniteSpiException - If registration failed.
  - unregister
```
public boolean unregister(String rsrcName)
```
    Unregisters all class loaders that have a class with given name or have a class with give ComputeTaskName value.
    
    Specified by:
    
    unregister in interface DeploymentSpi
    
    Parameters:
    
    rsrcName - Either class name or ComputeTaskName value for a class whose class loader needs to be unregistered.
    
    Returns:
    
    True if resource was unregistered.
  - setName
```
public IgniteSpiAdapter setName(String name)
```
    Sets SPI name.
    
    Overrides:
    
    setName in class IgniteSpiAdapter
    
    Parameters:
    
    name - SPI name.
    
    Returns:
    
    this for chaining.
  - toString
```
public String toString()
```
    Overrides:
    
    toString in class Object

Class UriDeploymentSpi

GAR file

URI

Code Example

Configuration

Protocols

File

File URI Example

HTTP/HTTPS

HTTP URI Example

Java Example

Spring Example

Field Summary

Fields inherited from class org.apache.ignite.spi.IgniteSpiAdapter

Constructor Summary

Method Summary

Methods inherited from class org.apache.ignite.spi.IgniteSpiAdapter

Methods inherited from class java.lang.Object

Methods inherited from interface org.apache.ignite.spi.IgniteSpi

Field Detail

DFLT_DEPLOY_DIR

XML_DESCRIPTOR_PATH

DEPLOY_TMP_ROOT_NAME

Constructor Detail

UriDeploymentSpi

Method Detail

setTemporaryDirectoryPath

setUriList

setCheckMd5

isCheckMd5

setEncodeUri

getTemporaryDirectoryPath

getUriList

setListener

getScanners

setScanners

spiStop

spiStart

findResource

register

unregister

setName

toString