Petals Documentation > Petals ESB > Petals Components > Components Use Cases > Use Cases for Petals-SE-Talend > Data Flow - From Petals to a Job using Attachments

Data Flow - From Petals to a Job using Attachments

Preliminary notes

This use case can be reproduced with both Talend Open Studio and Talend Integration Suite.
It is an alternative way to undertake what is done in the use case "Data Flow - From Petals to a job using a tPetalsInput".

Rationale

Send a CSV file as an attachment through Petals to a Talend job.
The job loads the content of the attached file and inserts its content into a database.
In the scope of this use case, we could imagine, even if it is not shown, that the file's content is transformed before being inserted.

The input message provides the attachment file.
Only the job's result is expected in the response.
Table of contents
Contributors

Creating and exporting the job

The job to be executed performs the following actions:

  1. The job is passed the file whose content must be inserted. The file is passed as an attachment to the message.
  2. The job loads the file's content.
  3. The job connects to a database and writes the data in this database.

This job has one context variable, pointing to the location of the file whose content must be loaded.

Creating the job

The job is made up of two components:

  1. The tFileInputDelimited reads the CSV file's content.
  2. The tMySqlOutput writes the data into the database.

Here is the overall aspect of the job.


Here is the schema of the tFileInputDelimited component.


Here are the properties of the tFileInputDelimited component.


Here are the properties of the tMySqlOutput component.

Exporting the job

Select the job and right-click it. Select Export Job Scripts.
In the Export type combo, select Petals ESB.
Update the target destination and let the job be exposed as a singleton.

Click Edit the exposed contexts.
A dialog shows up. Click the Export mode column, and select Parameter in the combo box. Click OK.

You should have the following dialog:

The link label should be updated and indicate the number of exported contexts.
Click Finish.

Deploying and testing in Petals

Looking at the generated WSDL

In the created Petals service assembly, the most interesting thing to look at is the WSDL.
Indeed, the WSDL will determine the way the exported service will be called.


The input message's description only expects the input attachment.

<xs:element name="executeJob" type="tns:executeJob" />
<xs:complexType name="executeJob">
	<xs:sequence>
		<xs:element minOccurs="0" name="contexts" type="tns:talendContexts" />
		<xs:element minOccurs="0" name="in-attachments" type="tns:inAttachments" />
		<xs:element maxOccurs="unbounded" minOccurs="0" name="in-data-bean" type="tns:inRow" />
		<xs:element maxOccurs="unbounded" minOccurs="0" name="talend-option" type="xs:string" />
	</xs:sequence>
</xs:complexType>

<xs:complexType name="talendContexts">
	<xs:sequence>
	</xs:sequence>
</xs:complexType>

<xs:complexType name="inAttachments">
	<xs:sequence>
		<xs:element name="outputLocation" nillable="true" type="tns:attachment" />
	</xs:sequence>
</xs:complexType>

<xs:complexType name="inRow">
	<xs:sequence>
	</xs:sequence>
</xs:complexType>


And the output message only includes the job's result.

<xs:element name="executeJobResponse" type="tns:executeJobResponse" />
<xs:complexType name="executeJobResponse">
	<xs:sequence>
		<xs:element minOccurs="0" name="talend-job-output" type="tns:talendJobOutput" />
	</xs:sequence>
</xs:complexType>

<xs:complexType name="talendJobOutput">
	<xs:sequence>
		<xs:element maxOccurs="unbounded" minOccurs="0" name="executionResult" nillable="true" type="ns1:stringArray" />
		<xs:element minOccurs="0" name="outAttachment" type="tns:outAttachments" />
		<xs:element maxOccurs="unbounded" minOccurs="0" name="outDataBean" nillable="true" type="tns:outRow" />
	</xs:sequence>
</xs:complexType>

<xs:complexType name="outAttachments">
	<xs:sequence>
	</xs:sequence>
</xs:complexType>

<xs:complexType name="outRow">
	<xs:sequence>
	</xs:sequence>
</xs:complexType>

Deploying and testing this new service

Since we use attachments, we will prefer using a Java client instead of SoapUI.
However, it is possible to use SoapUI to send or receive messages with attachments. Just make sure the MTOM property is activated.


The first thing to do is to create a service-unit for the Petals-BC-SOAP component, that exposes (consumes) our Talend job as a service outside the bus.
This step is not described here. You can take a look at the Petals-BC-SOAP documentation and the Petals Studio documentation.
Just make sure the SOAP configuration uses the InOut MEP.


Then, to generate a client from the WSDL, you can use Apache CXF or Axis2.
As an example, the following code was generated with Apache CXF 2.2.6.
Check the CXF documentation to see how to develop a service consumer in Java.


Only the main client class is shown here.
The client's code is the following:

package org.ow2.petals.talend;

/**
 * Please modify this class to meet your needs This class is not complete
 */

import java.io.File;
import java.net.MalformedURLException;
import java.net.URL;

import javax.activation.DataHandler;
import javax.activation.FileDataSource;
import javax.xml.namespace.QName;
import javax.xml.ws.BindingProvider;
import javax.xml.ws.soap.SOAPBinding;

import net.java.dev.jaxb.array.StringArray;

/**
 * This class was generated by Apache CXF 2.2.6 Mon Mar 29 18:38:51 CEST 2010
 * Generated source version: 2.2.6
 */
public final class Client {

	private static final QName SERVICE_NAME = new QName(
			"http://petals.ow2.org/talend/", "CsvToDatabaseService" );


	/**
	 * Constructor.
	 */
	private Client() {
		// nothing
	}


	/**
	 * Main method.
	 * @param args
	 * @throws Exception
	 */
	public static void main( String args[] ) throws Exception {

		URL wsdlURL = CsvToDatabaseService.WSDL_LOCATION;
		if( args.length > 0 ) {
			File wsdlFile = new File( args[ 0 ]);
			try {
				if( wsdlFile.exists()) {
					wsdlURL = wsdlFile.toURI().toURL();
				}
				else {
					wsdlURL = new URL( args[ 0 ]);
				}
			} catch( MalformedURLException e ) {
				e.printStackTrace();
			}
		}

		CsvToDatabaseService ss = new CsvToDatabaseService( wsdlURL, SERVICE_NAME );
		CsvToDatabaseServicePortType port = ss.getCsvToDatabaseEndpoint();

		// Activate the MTOM mode
		((SOAPBinding) ((BindingProvider) port).getBinding()).setMTOMEnabled( true );
		//

		// Set the input attachment
		File f = new File( "C:/Documents and Settings/vzurczak/Bureau/in.csv" );
		DataHandler dh = new DataHandler( new FileDataSource( f ));
		Attachment att = new Attachment();
		att.setFileContent( dh );

		InAttachments ia = new InAttachments();
		// Notice: outputLocation is the name of the context in the job.
		// The Talend export propagated the name in the service's WSDL.
		ia.setOutputLocation( att );
		//

		{
			System.out.println( "Invoking executeJob..." );
			org.ow2.petals.talend.TalendContexts contexts = null;
			java.util.List<org.ow2.petals.talend.InRow> inDataBean = null;
			java.util.List<java.lang.String> talendOption = null;
			try {
				org.ow2.petals.talend.TalendJobOutput result = port
						.executeJob( contexts,
								ia,
								inDataBean,
								talendOption );

				if( result == null )
					System.out.println( "Something went wrong, the result is null." );
				else {
					// The job's result
					System.out.println( "Job's result = " );

					int cpt = 0;
					for( StringArray row : result.getExecutionResult()) {
						System.out.print( "Row " + cpt++ + ": " );
						for( String col : row.getItem())
							System.out.print( col + " ; " );
					}
				}

			} catch( TalendTechnicalException_Exception e ) {
				System.out.println( "Expected exception: TalendTechnicalException has occurred." );
				System.out.println( e.toString());

			} catch( TalendBusinessException_Exception e ) {
				System.out.println( "Expected exception: TalendBusinessException has occurred." );
				System.out.println( e.toString());
			}
		}

		System.exit( 0 );
	}
}


Notice that the MTOM-mode was activated. Not enabling it will result in errors.


The execution output (displayed in the console) is:

Invoking executeJob...
Job's result =
Row 0: 0 ;

If the job execution fails, the 0 is replaced by another integer, e.g. 1.
One failure reason can be, as an example, that the database is not started.

To determine the act cause of a problem, you would have to use logging features available in the Talend design space.
However, let's make it clear, the job's logs are managed independently of Petals and its monitoring capabilities.

Labels

petals petals Delete
tutorial tutorial Delete
se se Delete
talend talend Delete
Enter labels to add to this page:
Please wait 
Looking for a label? Just start typing.