This version must be installed on [Petals ESB 5.3.0]+ Feature The SFTP component is a Binding Component which supports SFTP commands. This component provide a set of 5 operations: put: Write the received JBI messages in a file with a specified name. mput: Write the JBI message attachments it receives into files. get: Retrieve Xml file corresponding to a pattern, from a folder. Return it as message payload. getAsAttachment: Retrieve the first file corresponding to a pattern, from a folder. Return it as message's attachment.(XOP standard) mGet: Retrieve files corresponding to a pattern, from a folder. Return them as message's attachment. del: Delete one file. dir: List the content of a folder. These operations are described in an embedded WSDL. All attachment are processed by using XOP rules (XML-binary Optimized Packaging : http://www.w3.org/TR/xop10/).

Table of contents 1 Feature 2 Exposing a FTP connection as a service 2.1 Usage 2.1.1 Put Operation 2.1.2 MPut Operation 2.1.3 Get Operation 2.1.4 GetAsAttachment Operation 2.1.5 MGet Operation 2.1.6 Dir Operation 2.1.7 Del Operation 2.2 Configuration 2.2.1 CDK parameters defining service provider implementation 2.2.2 CDK parameters driving interceptors 2.2.3 Dedicated configuration 2.2.3.1 Connection parameters 2.2.3.2 Other parameters 2.2.4 Service unit content 2.2.5 Example 3 The integration service 4 Configuring the component 4.1 CDK parameters 4.2 Interception configuration 4.3 Dedicated configuration 5 Business monitoring 5.1 MONIT traces 5.2 Flow tracing activation 6 Monitoring the component 6.1 Using metrics 6.1.1 Common metrics 6.1.2 Dedicated metrics 6.2 Receiving alerts 6.2.1 Common alerts 6.2.2 Dedicated alerts 7 Troubleshooting 7.1 Try to connect to the SFTP server using the same SSH stack 7.2 Getting log from the SSH layer Contributors No contributors found for: authors on selected page(s)

Exposing a FTP connection as a service

The component is able to expose an external service in the JBI environment to send SFTP commands on an SFTP server (Service-unit mode).

Usage

The operations of the service exposed are explained here after:

• an operation need to be call explicitly in the form : {service-namespace}operation (example: {http://petals.ow2.org/components/sftp/version-1}put)
• all operations support wild card character.

Put Operation

When the put operation is set on the incoming IN message, the component write the XML message to the SFTP server.

The IN message looks like:

<?xml version="1.0" encoding="UTF-8"?>
<ver:put xmlns:ver="http://petals.ow2.org/components/sftp/version-1">
    <ver:body>xml body</ver:body>
    <ver:filename>file name on the  server</ver:filename>
</ver:put>

The service does not return a message.

The service might return an error when an element in the request is missing or if the SFTP connection failed.

The available exchange pattern is: InOnly.

The operation 'put' of the generic service supports the automatic creation of the remote directory if it does not exist. Just set the parameter 'create-folder' available in the section 'connection' to 'true'. See connection parameters.

MPut Operation

When the mput operation is set on the incoming IN message, the component write the attachments on the SFTP server.

The IN message looks like:

<?xml version="1.0" encoding="UTF-8"?>
<ver:mput xmlns:ver="http://petals.ow2.org/components/sftp/version-1">
<ver:attachments>
    <ver:filename><xop:Include xmlns:xop="http://www.w3.org/2004/08/xop/include" href="cid:test1.xml"/></ver:filename>
    <ver:filename name="my-file.txt"><xop:Include xmlns:xop="http://www.w3.org/2004/08/xop/include" href="cid:test2.xml"/></ver:filename>
</ver:attachments>
</ver:mput>

The file names used to store files on SFTP server is defined by the attribute 'name'. If this attribute is missing or empty, the file name will be set to the attachment identifier extracted from 'xop:Include#href'.

The service does not return a message.

The service might return an error when an element in the request is missing or if the SFTP connection failed.

The available exchange pattern is: InOnly.

The operation 'mput' of the generic service supports the automatic creation of the remote directory if it does not exist. Just set the parameter 'create-folder' available in the section 'connection' to 'true'. See connection parameters.

Get Operation

When the get operation is set on the incoming IN message, the component retrieve ONE file from the SFTP server and return it as an XML message content.

The name of the file to retrieve is set in the XML IN content of the message.

The IN message looks like:

<?xml version="1.0" encoding="UTF-8"?>
<ver:get xmlns:ver="http://petals.ow2.org/components/sftp/version-1">
     <ver:filename>file name to retrieve</ver:filename>
 </ver:get>

The OUT message returned to the consumer is the content of the XML file.

The service might return a Fault when an element in the request is missing or if the SFTP connection failed.

The available exchange patterns is: InOut.

Only xml file could be processed otherwise an error will be raised.

GetAsAttachment Operation

When the get operation is set on the incoming IN message, the component retrieve ONE file from the SFTP server and return it as an attachment.

The name of the file to retrieve is set in the XML IN content of the message.

The IN message looks like:

<?xml version="1.0" encoding="UTF-8"?>
<ver:getAsAttachment xmlns:ver="http://petals.ow2.org/components/sftp/version-1">
     <ver:filename>file name to retrieve</ver:filename>
</ver:getAsAttachment>

The OUT message returned to the consumer contains files, as attachments, and an XML message report:

<?xml version="1.0" encoding="UTF-8"?>
<tns:getAsAttachmentResponse xmlns:tns="http://petals.ow2.org/components/sftp/version-1">
  <tns:attachment>
    <tns:filename><xop:Include xmlns:xop="http://www.w3.org/2004/08/xop/include" href="cid:test1.xml"/></tns:filename>
    <tns:filename><xop:Include xmlns:xop="http://www.w3.org/2004/08/xop/include" href="cid:test2.xml"/></tns:filename>
  </tns:attachment>
</tns:getAsAttachmentResponse>

The service might return a Fault when an element in the request is missing or if the SFTP connection failed.

The available exchange pattern is: InOut.

MGet Operation

When the mget operation is set on the incoming IN message, the component retrieves files from the SFTP server, according to the file names (or filters) set in the XML request.

There is no recursivity, sub folders are ignored.

Each file is set in the OUT message as an attachment. The IN message looks like:

<?xml version="1.0" encoding="UTF-8"?>
<ver:mget xmlns:ver="http://petals.ow2.org/components/sftp/version-1">
    <!--1 or more repetitions:-->
    <ver:filename>*.xml</ver:filename>
    <ver:filename>myFile.txt</ver:filename>
</ver:mget>

The OUT message returned to the consumer contains files, as attachments, and an XML message report:

<?xml version="1.0" encoding="UTF-8"?>
<tns:mgetResponse xmlns:tns="http://petals.ow2.org/components/sftp/version-1">
  <tns:attachments>
    <tns:filename><xop:Include xmlns:xop="http://www.w3.org/2004/08/xop/include" href="cid:test1.xml"/></tns:filename>
    <tns:filename><xop:Include xmlns:xop="http://www.w3.org/2004/08/xop/include" href="cid:test2.xml"/></tns:filename>
  </tns:attachments>
</tns:mgetResponse>

The service might return a Fault when an element in the request is missing or if the SFTP connection failed.

The available exchange pattern is: InOut.

Dir Operation

When the dir operation is set on the incoming IN message, the component returns the file names listed from the SFTP
server.

The IN message looks like:

<?xml version="1.0" encoding="UTF-8"?>
<ver:dir xmlns:ver="http://petals.ow2.org/components/sftp/version-1 />

The OUT message returned to the consumer is defined as follow:

<tns:dirResponse xmlns:tns="http://petals.ow2.org/components/sftp/version-1">
         <!--Zero or more repetitions:-->
         <tns:filename>file1.txt</tns:filename>
         <tns:filename>linux.jpg</tns:filename>
</tns:dirResponse>

The service might return a Fault when an element in the request is missing or if the SFTP connection failed.

The available exchange pattern is: InOut.

Del Operation

When the del operation is set on the incoming IN message, the component delete the remote file.

The IN message looks like:

<?xml version="1.0" encoding="UTF-8"?><ver:del xmlns:ver="http://petals.ow2.org/components//version-3>
  <ver:filename>?</ver:filename>
</ver:del>

The service does not return a message.

The service might return an Error when an element in the request is missing or if the SFTP connection failed.

The available exchange pattern is: InOnly.

Configuration

All needed information must be defined in the service-unit JBI descriptor. This JBI descriptor is configured through parameters divided in following groups:

• JBI parameters that defines the service provider identification,
• CDK parameters that are parameters driving the service provider implementation at CDK layer,
• CDK interceptor parameters that are parameters driving interceptors at CDK layer,
• Dedicated parameters that are parameters driving the service provider implementation at component layer.

CDK parameters defining service provider implementation

The following parameters correspond to the CDK configuration of the service provider implementation.

The service provider is defined into the section 'provides' of the JBI descriptor, containing:

CDK parameters driving interceptors

The following parameters drive interceptors at CDK layer.

Interceptors can be defined to inject some post or pre-processing in the service provider processing or service consumer processing.

Using interceptor is very sensitive and must be manipulated only by power users. A non properly coded interceptor engaged in a component can lead to uncontrolled behaviors, out of the standard process.

Example of an interceptor configuration:

<?xml version="1.0" encoding="UTF-8"?>
<jbi:jbi xmlns:jbi="http://java.sun.com/xml/ns/jbi" xmlns:petalsCDK="http://petals.ow2.org/components/extensions/version-5">
   <jbi:services>
      <jbi:provides|consumes>
         <!--...-->
         <petalsCDK:su-interceptors>
            <petalsCDK:send>
               <petalsCDK:interceptor name="myInterceptorName">
                  <petalsCDK:param name="myParamName">myParamValue</petalsCDK:param>
                  <petalsCDK:param name="myParamName2">myParamValue2</petalsCDK:param>
               </petalsCDK:interceptor>
            </petalsCDK:send>
            <petalsCDK:accept>
               <petalsCDK:interceptor name="myInterceptorName">
                  <petalsCDK:param name="myParamName">myParamValue</petalsCDK:param>
               </petalsCDK:interceptor>
            </petalsCDK:accept>
            <petalsCDK:send-response>
               <petalsCDK:Interceptor name="myInterceptorName">
                  <petalsCDK:param name="myParamName">myParamValue</petalsCDK:param>
               </petalsCDK:Interceptor>
            </petalsCDK:send-response>
            <petalsCDK:accept-response>
               <petalsCDK:Interceptor name="myInterceptorName">
                  <petalsCDK:param name="myParamName">myParamValue</petalsCDK:param>
               </petalsCDK:Interceptor>
            </petalsCDK:accept-response>
         </petalsCDK:su-interceptors>
         <!--...-->
      </jbi:provides|consumes>
      <!--...-->
   </jbi:services>
</jbi:jbi>

Dedicated configuration

The following parameters correspond to the component specific configuration of the service provider implementation, divided in two groups:

• connection parameters,
• other parameters.

Connection parameters

Other parameters

Service unit content

The service unit has to contain the following elements, packaged in the archive:

• the META-INF/jbi.xml descriptor file as described above,
• it is also highly recommended to provide a WSDL description for service provider embedded in the service-unit. The service provider contract must implement the interface "{http://petals.ow2.org/components/sftp/version-1}SFTP" defined in the abstract WSDL 'SFtpServiceInterface.wsdl' available as resource in the component archive.

service-unit.zip
  + META-INF
    - jbi.xml (as defined above)
  - service.wsdl (recommended)

Example

An example of a Service Unit descriptor to provide an SFTP service:

<?xml version="1.0" encoding="UTF-8"?>
<jbi:jbi version="1.0"
    xmlns:generatedNs="http://petals.ow2.org/components/sftp/version-1"
    xmlns:jbi="http://java.sun.com/xml/ns/jbi"
    xmlns:petalsCDK="http://petals.ow2.org/components/extensions/version-5"
    xmlns:sftp="http://petals.ow2.org/components/sftp/version-1"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">

    <jbi:services binding-component="true">
        <jbi:provides
            interface-name="generatedNs:SftpInterface"
            service-name="generatedNs:SftpService"
            endpoint-name="SftpServiceEndpoint">

            <!-- CDK specific elements -->
            <petalsCDK:timeout>30000</petalsCDK:timeout>
            <petalsCDK:validate-wsdl>true</petalsCDK:validate-wsdl>
            <petalsCDK:forward-security-subject>false</petalsCDK:forward-security-subject>
            <petalsCDK:forward-message-properties>false</petalsCDK:forward-message-properties>
            <petalsCDK:forward-attachments>false</petalsCDK:forward-attachments>
            <petalsCDK:wsdl>SftpService.wsdl</petalsCDK:wsdl>

            <!-- Component specific elements -->
            <sftp:host>192.168.1.206</sftp:host>
            <sftp:port>22</sftp:port>
            <sftp:user>user1</sftp:user>

            <sftp:password>test</sftp:password>
            <!-- Authentication is done either by password or by privatekey/passphrase
            <sftp:privatekey></sftp:privatekey>
            <sftp:passphrase></sftp:passphrase>-->
        </jbi:provides>
    </jbi:services>
</jbi:jbi>

The integration service

The component exposes directly a generic SFTP service without deploying a service unit (Integration service mode). This generic service can be deactivate removing the file 'component.wsdl' from the component archive.

In integration service mode, parameters for the SFTP connection are set in the XML message request in a connection node added to the operation payload as following. See connection parameters for more details:

<?xml version="1.0" encoding="UTF-8"?>
<sftp:... xmlns:sftp="http://petals.ow2.org/components/sftp/generic/version-1">
   <sftp:connection>
      <sftp:server>?</ver:server>
      <sftp:port>?</ver:port>
      <sftp:user>?</ver:user>

      <!--Optional:-->
      <sftp:password>?</sftp:password>
      <sftp:folder>?</sftp:folder>
      <sftp:passphrase>?</sftp:passphrase>
      <sftp:private-key>?</sftp:private-key>
      <sftp:overwrite>?</sftp:overwrite>
      <sftp:create-folder>?</sftp:create-folder>
  </ver:connection>
</sftp:...>

Example of a PUT request:

<sftp:put xmlns:sftp="http://petals.ow2.org/components/sftp/generic/version-1">
   <sftp:body>xml body</sftp:body>
   <sftp:filename>file name on the  server</sftp:filename>
   
   <sftp:connection>
      <sftp:host>192.168.1.206</sftp:host>
      <sftp:port>22</sftp:port>
      <sftp:user>user1</sftp:user>
      <sftp:password>test</sftp:password>
   </sftp:connection>
</sftp:put>

Configuring the component

The component can be configured through the parameters of its JBI descriptor file. These parameters are divided in following groups:

• JBI parameters that have not to be changed otherwise the component will not work,
• CDK parameters that are parameters driving the processing of the CDK layer,
• Dedicated parameters that are parameters specific to this component.

CDK parameters

The component configuration includes the configuration of the CDK. The following parameters correspond to the CDK configuration.

• Installation: The parameter can be set during the installation of the component, by using the installation MBean (see JBI specifications for details about the installation sequence). If the parameter is optional and has not been defined during the development of the component, it is not available at installation time.
• Runtime: The paramater can be set during the installation of the component and during runtime. The runtime configuration can be changed using the CDK custom MBean named RuntimeConfiguration. If the parameter is optional and has not been defined during the development of the component, it is not available at installation and runtime times.

Interception configuration

Interceptors can be defined to inject some post or pre-processing in the component during service processing.

Using interceptor is very sensitive and must be manipulated only by power users. A non properly coded interceptor engaged in a component can lead to uncontrolled behaviors, out of the standard process.

Example of an interceptor configuration:

<?xml version="1.0" encoding="UTF-8"?>
<jbi:jbi xmlns:jbi="http://java.sun.com/xml/ns/jbi" xmlns:petalsCDK="http://petals.ow2.org/components/extensions/version-5" ...>
   <jbi:component>
      <!--...-->
      <petalsCDK:component-interceptors>
         <petalsCDK:interceptor active="true" class="org.ow2.petals.myInterceptor" name="myInterceptorName">
            <petalsCDK:param name="myParamName">myParamValue</petalsCDK:param>
            <petalsCDK:param name="myParamName2">myParamValue2</petalsCDK:param>
         </petalsCDK:interceptor>
      </petalsCDK:component-interceptors>
      <!--...-->
   </jbi:component>
</jbi:jbi>

Dedicated configuration

No dedicated configuration parameter is available.

Business monitoring

MONIT traces

Each service provider exposed in Petals ESB is able to log MONIT traces with following information:

• on service provider invocation, when receiving an incoming request, with following attributes:
  • traceCode set to provideFlowStepBegin,
  • flowInstanceId set to the flow instance identifier retrieved from the incoming request,
  • flowStepId set to an UUID value,
  • flowStepInterfaceName set to the service provider interface name,
  • flowStepServiceName set to the service provider service name,
  • flowStepOperationName set to the operation of the invoked service provider,
  • flowStepEndpointName set to the service provider endpoint name,
  • flowPreviousStepId set to the step identifier of the previous step, retrieved from the incoming request.
• on external service provider invocation, when sending a request to the external service provider, with following attributes:
  • traceCode set to provideExtFlowStepBegin,
  • flowInstanceId set to the flow instance identifier retrieved from the incoming request,
  • flowStepId set to an UUID value,
  • flowPreviousStepId set to the step identifier of the previous step that is the internal processing of the incoming request into the BC.
• on external service provider termination, when receiving the outgoing response of the external service provider, with following attributes:
  • traceCode set to provideExtFlowStepEnd or provideExtFlowStepFailure,
  • flowInstanceId set to the flow instance identifier retrieved from the incoming request,
  • flowStepId set to the flow step identifier defined for the external service provider invocation.
• on service provider termination, when returning the outgoing response, with following attributes:
  • traceCode set to provideFlowStepEnd or provideFlowStepFailure,
  • flowInstanceId set to the flow instance identifier retrieved from the incoming request,
  • flowStepId set to the flow step identifier defined on incoming request receipt.

Flow tracing activation

The flow tracing (ie. MONIT traces generation) is defined according to the property 'org.ow2.petals.monitoring.business.activate-flow-tracing' of the incoming request. If the property does not exist, the parameter activate-flow-tracing of the service provider definition will be inspected. If no parameter is defined at service provider level, the component configuration parameter 'activate-flow-tracing' is used. Finally, by default, the flow tracing is enabled.

Monitoring the component

Using metrics

Several probes providing metrics are included in the component, and are available through the JMX MBean 'org.ow2.petals:type=custom,name=monitoring_<component-id>', where <component-id> is the unique JBI identifier of the component.

Common metrics

The following metrics are provided through the Petals CDK, and are common to all components:

Dedicated metrics

No dedicated metric is available.

Receiving alerts

Several alerts are notified by the component through notification of the JMX MBean 'org.ow2.petals:type=custom,name=monitoring_<component-id>', where <component-id> is the unique JBI identifier of the component.

To integrate these alerts with Nagios, see Receiving Petals ESB defects in Nagios.

Common alerts

Dedicated alerts

No dedicated alert is available.

Troubleshooting

Try to connect to the SFTP server using the same SSH stack

If you have problem establishing a connection with the SSH server, you can test your parameters with the Petals BC SFTP tooling of Petals CLI 2.1.1 or upper.
Note: The Petals CLI command use the same SSH stack than the component.

Getting log from the SSH layer

The BC SFTP embeds J2SSH as SSH client stack. You can configure the log of the SSH stack extending the Petals ESB logging configuration with:

# The full J2SSH
com.sshtools.j2ssh.level=INFO

# J2SSH by parts
com.sshtools.j2ssh.sftp.level=INFO
com.sshtools.j2ssh.transport.level=WARNING

Caution the J2SSH layer is verbose at INFO level !