Petals Documentation > Petals ESB > Petals Components > Components User Guide > Petals-BC-SFTP > Petals-BC-SFTP 1.6.1+

Petals-BC-SFTP 1.6.1+

This version must be installed on Petals ESB 5.0.1+

Feature

The SFTP component is a Binding Component which supports SFTP commands.

This component provide a set of 5 operations:

  • put: Write the received JBI messages in a file with a specified name.
  • mput: Write the JBI message attachments it receives into files.
  • get: Retrieve Xml file corresponding to a pattern, from a folder. Return it as message payload.
  • getAsAttachment: Retrieve the first file corresponding to a pattern, from a folder. Return it as message's attachment.(XOP standard)
  • mGet: Retrieve files corresponding to a pattern, from a folder. Return them as message's attachment.
  • del: Delete one file.
  • dir: List the content of a folder.

These operations are described in an embedded WSDL.

All attachment are processed by using XOP rules.(XML-binary Optimized Packaging : http://www.w3.org/TR/xop10/ )
Table of contents
Contributors
No contributors found for: authors on selected page(s)

Exposing a SFTP connection as a service

In provide mode, the component expose an external service in the JBI environment to send SFTP commands on an SFTP server.

The component can expose directly a generic SFTP service without deploying a service unit (Generic mode) or use a service-unit (Service-unit mode).

In generic mode, parameter for the SFTP connection are set in the XML message request in a connection node (see Generic Mode configuration).

Usage

When using the SFTP service, you can optionnaly define the SFTP connection information in your XML request. If you don't, the service unit parameters are used.

  • All operations support wild card character

Put Operation

When the put operation is set on the incoming IN message, the component write the XML message to the SFTP server.

The IN message looks like :

<?xml version="1.0" encoding="UTF-8"?>
<ver:put xmlns:ver="http://petals.ow2.org/components/sftp/version-1">
    <ver:body>xml body</ver:body>
    <ver:filename>file name on the  server</ver:filename>
</ver:put>

The service does not return a message.

The service might return an error when an element in the request is missing or if the SFTP connection failed.

The available exchange pattern is : InOnly.

MPut Operation

When the mput operation is set on the incoming IN message, the component write the attachments on the SFTP server.

The IN message looks like :

<?xml version="1.0" encoding="UTF-8"?>
<ver:mput xmlns:ver="http://petals.ow2.org/components/sftp/version-1">
<ver:attachments>
    <ver:filename><xop:Include xmlns:xop="http://www.w3.org/2004/08/xop/include" href="cid:test1.xml"/></ver:filename>
    <ver:filename><xop:Include xmlns:xop="http://www.w3.org/2004/08/xop/include" href="cid:test2.xml"/></ver:filename>
</ver:attachments>
</ver:mput>

The service does not return a message.

The service might return an error when an element in the request is missing or if the SFTP connection failed.

The available exchange pattern is : InOnly.

Get Operation

When the get operation is set on the incoming IN message, the component retrieve ONE file from the SFTP server and return it as an XML message content.

The name of the file to retrieve is set in the XML IN content of the message.

The IN message looks like : 

<?xml version="1.0" encoding="UTF-8"?>
<ver:get xmlns:ver="http://petals.ow2.org/components/sftp/version-1">
     <ver:filename>file name to retrieve</ver:filename>
 </ver:get>

The OUT message returned to the consumer is the content of the XML file

The service might return a Fault when an element in the request is missing or if the SFTP connection failed

The available exchange patterns is : InOut.

Only xml file could be processed otherwise an error will be raised.

GetAsAttachment Operation

When the get operation is set on the incoming IN message, the component retrieve ONE file from the SFTP server and return it as an attachment.

The name of the file to retrieve is set in the XML IN content of the message.

The IN message looks like :

<?xml version="1.0" encoding="UTF-8"?>
<ver:getAsAttachment xmlns:ver="http://petals.ow2.org/components/sftp/version-1">
     <ver:filename>file name to retrieve</ver:filename>
</ver:getAsAttachment>

The OUT message returned to the consumer contains files, as attachments, and an XML message report :

<?xml version="1.0" encoding="UTF-8"?>
<tns:getAsAttachmentResponse xmlns:tns="http://petals.ow2.org/components/sftp/version-1">
  <tns:attachment>
    <tns:filename><xop:Include xmlns:xop="http://www.w3.org/2004/08/xop/include" href="cid:test1.xml"/></tns:filename>
    <tns:filename><xop:Include xmlns:xop="http://www.w3.org/2004/08/xop/include" href="cid:test2.xml"/></tns:filename>
  </tns:attachment>
</tns:getAsAttachmentResponse>

The service might return a Fault when an element in the request is missing or if the SFTP connection failed

The available exchange pattern is : InOut.

MGet Operation

When the mget operation is set on the incoming IN message, the component retrieves files from the SFTP server, according to the file names (or filters) set in the XML request.
There is no recursivity, sub folders are ignored.

Each file is set in the OUT message as an attachment. The IN message looks like : 

<?xml version="1.0" encoding="UTF-8"?>
<ver:mget xmlns:ver="http://petals.ow2.org/components/sftp/version-1">
    <!--1 or more repetitions:-->
    <ver:filename>*.xml</ver:filename>
    <ver:filename>myFile.txt</ver:filename>
</ver:mget>

The OUT message returned to the consumer contains files, as attachments, and an XML message report : 

<?xml version="1.0" encoding="UTF-8"?>
<tns:mgetResponse xmlns:tns="http://petals.ow2.org/components/sftp/version-1">
  <tns:attachments>
    <tns:filename><xop:Include xmlns:xop="http://www.w3.org/2004/08/xop/include" href="cid:test1.xml"/></tns:filename>
    <tns:filename><xop:Include xmlns:xop="http://www.w3.org/2004/08/xop/include" href="cid:test2.xml"/></tns:filename>
  </tns:attachments>
</tns:mgetResponse>

The service might return a Fault when an element in the request is missing or if the SFTP connection failed

The available exchange pattern is : InOut.

Dir Operation

When the dir operation is set on the incoming IN message, the component returns the file names listed from the SFTP
server.
The IN message looks like :

<?xml version="1.0" encoding="UTF-8"?><ver:dir xmlns:ver="http://petals.ow2.org/components/sftp/version-1>
</ver:dir>

The OUT message returned to the consumer is defined as follow :

<tns:dirResponse xmlns:tns="http://petals.ow2.org/components/sftp/version-1">
         <!--Zero or more repetitions:-->
         <tns:filename>file1.txt</tns:filename>
         <tns:filename>linux.jpg</tns:filename>
</tns:dirResponse>

The service might return a Fault when an element in the request is missing or if the SFTP connection failed

The available exchange pattern is : InOut.

Del Operation

When the del operation is set on the incoming IN message, the component delete the remote file.
The IN message looks like :

<?xml version="1.0" encoding="UTF-8"?><ver:del xmlns:ver="http://petals.ow2.org/components//version-3>
  <ver:filename>?</ver:filename>
</ver:del>

The service does not return a message.
The service might return an Error when an element in the request is missing or if the SFTP connection failed.
The available exchange pattern is : InOnly.

Configuration

Generic Mode

To allow the component to provide his generic service, the component must have a wsdl with the name : component.wsdl. An example of this file is available in the component.

To deactivate the generic service supplies by the component,simply erase the file : component.wsdl.

To configure the SFTP connection in your XML message request, you have to provide a connection element. 

...
<!--Optional:-->
         <ver:connection>
            <ver:server>?</ver:server>
            <ver:port>?</ver:port>
            <ver:user>?</ver:user>

            <!--Optional:-->
            <ver:password>?</ver:password>
            <ver:folder>?</ver:folder>
            <ver:passphrase>?</ver:passphrase>
            <ver:private-key>?</ver:private-key>
         </ver:connection>
...

Example of a PUT request :

<sftp:put xmlns:sftp="http://petals.ow2.org/components/sftp/generic/version-1">
    <sftp:body>xml body</sftp:body>
    <sftp:filename>file name on the  server</sftp:filename>
   <!--Optional:-->
   <sftp:connection>
        <sftp:host>192.168.1.206</sftp:host>
        <sftp:port>22</sftp:port>
        <sftp:user>user1</sftp:user>
        <sftp:password>test</sftp:password>
   </sftp:connection>
</sftp:put>

Service Unit descriptor

Excample of a Service Unit descriptor to provide a SFTP service:

<?xml version="1.0" encoding="UTF-8"?>
<!--
    JBI descriptor for the Petals' "petals-bc-sftp" component (SFTP).
    Originally created for the version 1.2 of the component.
 -->
<jbi:jbi version="1.0"
    xmlns:generatedNs="http://petals.ow2.org/components/sftp/version-1"
    xmlns:jbi="http://java.sun.com/xml/ns/jbi"
    xmlns:petalsCDK="http://petals.ow2.org/components/extensions/version-5"
    xmlns:sftp="http://petals.ow2.org/components/sftp/version-1"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">

    <!-- Import a Service into Petals or Expose a Petals Service => use a BC. -->
    <jbi:services binding-component="true">

        <!-- Import a Service into Petals => provides a Service. -->
        <jbi:provides
            interface-name="generatedNs:SftpInterface"
            service-name="generatedNs:SftpService"
            endpoint-name="SftpServiceEndpoint">

            <!-- CDK specific elements -->
            <petalsCDK:timeout>30000</petalsCDK:timeout>
            <petalsCDK:validate-wsdl>true</petalsCDK:validate-wsdl>
            <petalsCDK:forward-security-subject>false</petalsCDK:forward-security-subject>
            <petalsCDK:forward-message-properties>false</petalsCDK:forward-message-properties>
            <petalsCDK:forward-attachments>false</petalsCDK:forward-attachments>
            <petalsCDK:wsdl>SftpService.wsdl</petalsCDK:wsdl>

            <!-- Component specific elements -->
            <sftp:host>192.168.1.206</sftp:host>
            <sftp:port>22</sftp:port>
            <sftp:user>user1</sftp:user>

            <sftp:password>test</sftp:password>
            <!-- Authentication is done either by password or by privatekey/passphrase
            <sftp:privatekey></sftp:privatekey>
            <sftp:passphrase></sftp:passphrase>-->
        </jbi:provides>
    </jbi:services>
</jbi:jbi>
Configuration of a Service Unit to provide a service (JBI)

Parameter Description
Default
Required
provides Describe the JBI service that will be exposed into the JBI bus. Interface (QName), Service (QName) and Endpoint (String) attributes are required. - Yes

Configuration of a Service Unit to provide a service (CDK)

Parameter Description
 Default
 Required
timeout Timeout in milliseconds of a synchronous send. This parameter is used by the method sendSync (Exchange exchange) proposes by the CDK Listeners classes.
Set it to 0 for an infinite timeout. 		30000 No
exchange-properties This sections defines the list of properties to set to the JBI exchange when processing a service. - No
message-properties This sections defines the list of properties to set to the JBI message when processing a service. - No
validate-wsdl Activate the validation of the WSDL when deploying a service unit. true No
wsdl
Path to the WSDL document describing services and operations exposed by the provided JBI endpoints defined in the SU.
The value of this parameter is :
  • an URL
  • a file relative to the root of the SU package
    If not specified, a basic WSDL description is automaticaly provided by the CDK.
- No
forward-attachments
Defines if attachment will be forwarded from IN message to OUT message.
false No
forward-message-properties
Defines if the message properties will be forwarded from IN message to OUT message. false No
forward-security-subject
Defines if the security subject will be forwarded from IN message to OUT message. false No

Configuration of a Service Unit to provide a service (SFTP)

Parameter Description Default Required
server IP or DNS name of the server
-
Yes
except when using Dynamic mode
port the port number of the server
21
No
user the user login name used to connect to the server
-
Yes
except when using Dynamic mode
password the user password
-
Yes
except when using Dynamic mode
folder the folder on the server
-
No
privatekey the path to the user's private key (on the PEtALS machine)
-
Yes
except when using dynamic mode or password authentication
passphrase the passphrase matching the given private key
-
No

Interceptor

Example of an interceptor configuration:

<?xml version="1.0" encoding="UTF-8"?>
<!--...-->
<petalsCDK:su-interceptors>
  <petalsCDK:send>
    <petalsCDK:interceptor name="myInterceptorName">
      <petalsCDK:param name="myParamName">myParamValue</petalsCDK:param>
      <petalsCDK:param name="myParamName2">myParamValue2</petalsCDK:param>
    </petalsCDK:interceptor>
  </petalsCDK:send>
  <petalsCDK:accept>
    <petalsCDK:interceptor name="myInterceptorName">
      <petalsCDK:param name="myParamName">myParamValue</petalsCDK:param>
    </petalsCDK:interceptor>
  </petalsCDK:accept>
  <petalsCDK:send-response>
    <petalsCDK:Interceptor name="myInterceptorName">
      <petalsCDK:param name="myParamName">myParamValue</petalsCDK:param>
    </petalsCDK:Interceptor>
  </petalsCDK:send-response>
  <petalsCDK:accept-response>
    <petalsCDK:Interceptor name="myInterceptorName">
      <petalsCDK:param name="myParamName">myParamValue</petalsCDK:param>
    </petalsCDK:Interceptor>
  </petalsCDK:accept-response>
</petalsCDK:su-interceptors>
<!--...-->

Interceptors configuration for SU (CDK)

Parameter Description Default Required
send Interceptor dedicated to send phase, for an exchange sent by a consumer - No
accept Interceptor dedicated to receive phase, for an exchange received by a provider - No
send-response Interceptor dedicated to send phase, for an exchange (a response) received by a consumer - No
accept-response Interceptor dedicated to receive phase, for an exchange sent (a response) by a provider - No
interceptor - name Logical name of the interceptor instance. It can be referenced to add extended parameters by a SU Interceptor configuration. - Yes
param[] - name The name of the parameter to use for the interceptor for this SU - No
param[] The value of the parameter to use for the interceptor for this SU - No

Service Unit content

The Service Unit has to contain the following elements, packaged in an archive:

  • The META-INF/jbi.xml descriptor file, has described above,
  • An optional wsdl file describing the related service
service-unit.zip
+ META-INF
    - jbi.xml (as defined above)
    - service.wsdl (optional)

Component Configuration

Configuration of the component, CDK part

Parameter Description Default Scope
acceptor-pool-size The size of the thread pool used to accept Message Exchanges from the NMR. Once a message is accepted, its processing is delegated to the processor pool thread. 1
 Runtime
acceptor-retry-number Number of tries to submit a message exchange to a processor for processing before to declare that it cannot be processed. 40
 Installation
acceptor-retry-wait Base duration, in milliseconds, to wait between two processing submission tries. At each try, the new duration is the previous one plus this base duration. 250
 Installation
acceptor-stop-max-wait The max duration (in milliseconds) before, on component stop, each acceptor is stopped by force. 500
 Runtime
processor-pool-size The size of the thread pool used to process Message Exchanges. Once a message is accepted, its processing is delegated to one of the thread of this pool. 10 Runtime
processor-max-pool-size The maximum size of the thread pool used to process Message Exchanges. The difference between this size and the processor-pool-size represents the dynamic threads that can be created and destroyed during overhead processing time.
50
 Runtime
processor-keep-alive-time When the number of processors is greater than the core, this is the maximum time that excess idle processors will wait for new tasks before terminating, in seconds.
300
 Runtime
processor-stop-max-wait The max duration (in milliseconds) of message exchange processing on stop phase (for all processors).
 15000
 Runtime
time-beetween-async-cleaner-runs The time (in milliseconds) between two runs of the asynchronous message exchange cleaner.
 2000
 Installation
properties-file Name of the file containing properties used as reference by other parameters. Parameters reference the property name using a placeholder in the following pattern ${myPropertyName}. At runtime, the expression is replaced by the value of the property.

The properties file can be reloaded using the JMX API of the component. The runtime configuration MBean provides an operation to reload these place holders. Check the service unit parameters that support this reloading.

The value of this parameter is :
  • an URL
  • a file relative to the PEtALS installation path
  • an absolute file path
  • an empty value to stipulate a non-using file.
 - Installation
monitoring-sampling-period Period, in seconds, of a sample used by response time probes of the monitoring feature.
 300
 Installation

Definition of CDK parameter scope :

  • Installation: The parameter can be set during the installation of the component, by using the installation MBean (see JBI specifications for details about the installation sequence). If the parameter is optional and has not been defined during the development of the component, it is not available at installation time.
  • Runtime: The paramater can be set during the installation of the component and during runtime. The runtime configuration can be changed using the CDK custom MBean named RuntimeConfiguration. If the parameter is optional and has not been defined during the development of the component, it is not available at installation and runtime times.

Interceptor

Interceptors can be defined to inject some post or pre processing in the component during service processing.

Using interceptor is very sensitive and must be manipulate only by power users. An non properly coded interceptor engaged in a component can lead to uncontrolled behaviors, out of the standard process.

Example of an interceptor configuration:

<?xml version="1.0" encoding="UTF-8"?>
<!--...-->
<petalsCDK:component-interceptors>
  <petalsCDK:interceptor active="true" class="org.ow2.petals.myInterceptor" name="myInterceptorName">
    <petalsCDK:param name="myParamName">myParamValue</petalsCDK:param>
    <petalsCDK:param name="myParamName2">myParamValue2</petalsCDK:param>
  </petalsCDK:interceptor>
</petalsCDK:component-interceptors>
<!--...-->

Interceptors configuration for Component (CDK)

Parameter Description Default Required
interceptor - class Name of the interceptor class to implement. This class must extend the abstract class org.ow2.petals.component.common.interceptor.Interceptor. This class must be loadable from the component classloader, or in a dependent Shared Library classloader. - Yes
interceptor - name Logical name of the interceptor instance. It can be referenced to add extended parameters by a SU Interceptor configuration. - Yes
interceptor - active If true, the Interceptor instance is activated for every SU deployed on the component.
If false, the Interceptor can be activated:
-by the InterceptorManager Mbean at runtime, to activate the interceptor for every deployed SU.
-by a SU configuration 		- Yes
param[] - name The name of the parameter to use for the interceptor. - No
param[] The value of the parameter to use for the interceptor. - No

Monitoring the component

Using metrics

Several probes providing metrics are included in the component, and are available through the JMX MBean 'org.ow2.petals:type=custom,name=monitoring_<component-id>', where <component-id> is the unique JBI identifier of the component.

Common metrics

The following metrics are provided through the Petals CDK, and are common to all components:

Metrics, as MBean attribute Description Detail of the value Configurable
MessageExchangeAcceptorThreadPoolMaxSize The maximum number of threads of the message exchange acceptor thread pool integer value, since the last startup of the component yes, through acceptor-pool-size
MessageExchangeAcceptorThreadPoolCurrentSize The current number of threads of the message exchange acceptor thread pool. Should be always equals to MessageExchangeAcceptorThreadPoolMaxSize. instant integer value no
MessageExchangeAcceptorCurrentWorking The current number of working message exchange acceptors. instant long value no
MessageExchangeAcceptorMaxWorking The max number of working message exchange acceptors. long value, since the last startup of the component no
MessageExchangeAcceptorAbsoluteDurations The aggregated durations of the working message exchange acceptors since the last startup of the component. n-tuple value containing, in nanosecond:
  • the maximum duration,
  • the average duration,
  • the minimum duration.
 no
MessageExchangeAcceptorRelativeDurations The aggregated durations of the working message exchange acceptors on the last sample. n-tuple value containing, in nanosecond:
  • the maximum duration,
  • the average duration,
  • the minimum duration,
  • the 10-percentile duration (10% of the durations are lesser than this value),
  • the 50-percentile duration (50% of the durations are lesser than this value),
  • the 90-percentile duration (90% of the durations are upper than this value).
 no
MessageExchangeProcessorAbsoluteDurations The aggregated durations of the working message exchange processor since the last startup of the component. n-tuple value containing, in milliseconds:
  • the maximum duration,
  • the average duration,
  • the minimum duration.
 no
MessageExchangeProcessorRelativeDurations The aggregated durations of the working message exchange processor on the last sample. n-tuple value containing, in milliseconds:
  • the maximum duration,
  • the average duration,
  • the minimum duration,
  • the 10-percentile duration (10% of the durations are lesser than this value),
  • the 50-percentile duration (50% of the durations are lesser than this value),
  • the 90-percentile duration (90% of the durations are upper than this value).
 no
MessageExchangeProcessorThreadPoolActiveThreadsCurrent The current number of active threads of the message exchange processor thread pool instant integer value no
MessageExchangeProcessorThreadPoolActiveThreadsMax The maximum number of threads of the message exchange processor thread pool that was active integer value, since the last startup of the component no
MessageExchangeProcessorThreadPoolIdleThreadsCurrent The current number of idle threads of the message exchange processor thread pool instant integer value no
MessageExchangeProcessorThreadPoolIdleThreadsMax The maximum number of threads of the message exchange processor thread pool that was idle integer value, since the last startup of the component no
MessageExchangeProcessorThreadPoolMaxSize The maximum size, in threads, of the message exchange processor thread pool instant integer value yes, through http-thread-pool-size-max
MessageExchangeProcessorThreadPoolMinSize The minimum size, in threads, of the message exchange processor thread pool instant integer value yes, through http-thread-pool-size-min
MessageExchangeProcessorThreadPoolQueuedRequestsCurrent The current number of enqueued requests waiting to be processed by the message exchange processor thread pool instant integer value no
MessageExchangeProcessorThreadPoolQueuedRequestsMax The maximum number of enqueued requests waiting to be processed by the message exchange processor thread pool since the last startup of the component instant integer value no
ServiceProviderInvocations The number of service provider invocations grouped by:
  • interface name, as QName, the invoked service provider,
  • service name, as QName, the invoked service provider,
  • invoked operation, as QName,
  • message exchange pattern,
  • and execution status (PENDING, ERROR, FAULT, SUCCEEDED).
 integer counter value since the last startup of the component no
ServiceProviderInvocationsResponseTimeAbs The aggregated response times of the service provider invocations since the last startup of the component grouped by:
  • interface name, as QName, the invoked service provider,
  • service name, as QName, the invoked service provider,
  • invoked operation, as QName,
  • message exchange pattern,
  • and execution status (PENDING, ERROR, FAULT, SUCCEEDED).
 n-tuple value containing, in millisecond:
  • the maximum response time,
  • the average response time,
  • the minimum response time.
 no
ServiceProviderInvocationsResponseTimeRel The aggregated response times of the service provider invocations on the last sample, grouped by:
  • interface name, as QName, the invoked service provider,
  • service name, as QName, the invoked service provider,
  • invoked operation, as QName,
  • message exchange pattern,
  • and execution status (PENDING, ERROR, FAULT, SUCCEEDED).
 n-tuple value containing, in millisecond:
  • the maximum response time,
  • the average response time,
  • the minimum response time,
  • the 10-percentile response time (10% of the response times are lesser than this value),
  • the 50-percentile response time (50% of the response times are lesser than this value),
  • the 90-percentile response time (90% of the response times are lesser than this value).
 no

Dedicated metrics

No dedicated metric is available.

Receiving alerts

Several alerts are notified by the component through notification of the JMX MBean 'org.ow2.petals:type=custom,name=monitoring_<component-id>', where <component-id> is the unique JBI identifier of the component.

To integrate these alerts with Nagios, see Receiving Petals ESB defects in Nagios.

Common alerts

Defect JMX Notification
A message exchange acceptor thread is dead
  • type: org.ow2.petals.component.framework.process.message.acceptor.pool.thread.dead
  • no user data
No more thread is available in the message exchange acceptor thread pool
  • type: org.ow2.petals.component.framework.process.message.acceptor.pool.exhausted
  • no user data
No more thread is available to run a message exchange processor
  • type: org.ow2.petals.component.framework.process.message.processor.thread.pool.exhausted
  • no user data

Dedicated alerts

No dedicated alert is available.

Troubleshooting

Try to connect to the SFTP server using the same SSH stack

If you have problem establishing a connection with the SSH server, you can test your parameters with the Petals BC SFTP tooling of Petals CLI 2.1.1 or upper.
Note: The Petals CLI command use the same SSH stack than the component.

Getting log from the SSH layer

The BC SFTP embeds J2SSH as SSH client stack. You can configure the log of the SSH stack extending the Petals ESB logging configuration with:

# The full J2SSH
com.sshtools.j2ssh.level=INFO

# J2SSH by parts
com.sshtools.j2ssh.sftp.level=INFO
com.sshtools.j2ssh.transport.level=WARNING
Caution the J2SSH layer is verbose at INFO level !
Enter labels to add to this page:
Please wait 
Looking for a label? Just start typing.