In my last Talend tutorial, I have covered how to use tMap component to reject rows from Input file. There is an alternative to it, instead of using tMap component, we can also use tFilterRow component to filter the records. I am going to use same scenario as used in the previous post and will try to achieve same output as we got in the last post.

Requirement 

In this demo, we are going to process an input file. This input file has only one column: NAME.We are going to pick records which start with only A or B and will reject rest of the records. The input file is as mentioned below:

The Final Job Diagram :




Steps For The Processing :

The first step is to read the input file. As used in the previous posts also, I am going to use tFileInputDelimited component to read the file. The configuration of this components is as below:



The next step is to process records based on our requirement. We have to analyze the first character of the input records and have to check whether it is equal to A or B. For the processing, we are going to use tFilterRow component. The whole configuration will look like this:


The code :

input_row.Name.substring(0,1).equals("A")||input_row.Name.substring(0,1).equals("B")

A template is already provided by Talend to use the filter condition in advanced mode. This will give you the hint how to call columns. This window gives us a lot of dynamism as we can write our own custom condition in Java and we don't have to rely only on the existing functions which are very few and basic for the filter criteria.

Please check this link to learn how to use Substring in Java.

tFilterRow Demystified..

tFilterRow provides us two mode of filtering : Simple and Advanced. The Simple Mode filtering requires you to select values from the given options. To add your columns and filter criteria, just click on the (+) button. But, you won't get so much options to choose from, the given options under Function and Operator are very basic. The next option is the Advanced Mode. This option add a lot of dynamism in the tFilterRow and makes it very powerful. Here, you can use your basic Java skills and add the criteria based on your will. If you want to use both mode, then select And condition from the dropdown under Logical Operator Used To Combine Conditions.

You can get more details from here : tFilterRow In Detail



Once you are done with the tFilterRow configuration, just link it to the desired output component. I am using tLogRow component here. Similarly, you can use tFileOutputDelimited to put the records in the text file. 


Also, you don't have to define anything specifically for the Reject Rows and to link it to the output component. First, click on the tFilterRow component, Go to Row-> Filter and connect first tLogRow component. Next, repeat the same process but this time Talend will only give you Reject option, use this to link other tLogRow component. 



Click on the Reject Rows to get more details about it. The Reject row comes with the default Error message column. 



The Final Job Execution and Output :

You will get an output like this once you run the Job :











We usually get the requirement where we have to pick some rows and have to drop others based on some conditions. This requires the processing of an input file. If you have worked on some ETL tools before like Datastage, then Transformer is your component for the processing. Same way,  Talend provides tMap component for processing. In this demo, we are going to filter rows using tMap component.

Let's check how to get it done...

Requirement 

In this demo, we are going to process an input file. This input file has only one column: CITY.
We are going to pick records which start with only A or B and will reject rest of the records. The input file is as mentioned below:



The Final Job Diagram:



 Steps For The Processing 

  • The first step is to read the input file. As used in the previous posts also, I am going to use tFileInputDelimited component to read the file. The configuration of this components is as below:

  • The next step is to process records based on our requirement. We have to analyze the first character of the input records and have to check whether it is equal to A or B. For the processing, we are going to use tMap component. The whole configuration will look like this:

(Click To Enlarge)

  • At the left side of the screen, we will get the input columns. As soon as we connect the input to the tMap, all the columns will get automatically populated.



  • At the right side of the screen, you can see two output window: out1 and out2. This is required as we are going to filter the records based on the condition. The out1 window is for the valid records ( The records that will match our criteria). The out2 window is for the rejected records.



  • To put the conditional statement, first of all, open the expression editor. Once you are inside the expression editor, you can use the different functions as provided by Talend. Here, I am using LEFT function under StringHandling categories.




  • To reject records, we don't have to put any condition specifically. Talend provides a feature to reject records and we are just using that. Just open the tMap settings and select the value as True for Catch Output Reject.

  • Finally, we just have to place records coming out from Ou1 and Out2 to an output file. For this, we are going to use tFileOutputDelimited component. The VALID.txt file will store the records which are matching our criteria and the rest will go to INVALID.txt.





The Final Job Execution:


It's time to execute our designed job. The final result would be like this:



© 2015 Techie's House. Designed by Bloggertheme9