Spring Batch is lightweight Spring based solution to process large number of records in batches and provides high performance batch jobs though optimization and partitioning techniques. In last post we saw How to create a Spring Batch project using STS. In this post we will see process of Spring Batch – Reading & Writing file and understand the code and configuration needed for it.
Input & Output Data
This is a spring batch demo application using the org.springframework.boot:spring-boot-starter-batch.The batch is pulling records from a database, process them and insert the processed records back into a new database table. Here I am going to show how to write data into the database using Spring Boot Batch. A step by step way to building spring boot batch example to read data from csv file and loading into mysql database using FlatFileItemReader, JdbcBatchItemWriter, StepBuilderFactory, StepBuilderFactory, JobBuilderFactory.
Change hard drive serial number vmware. I believe QuickBooks is one of them as it will fail to detect a valid license after being cloned from one hard drive to another.Changing the serial number of the hard drive (by this I mean in the firmware, not just the label) can also result in issues hardware bound software.Out of curiosity, did the person give you any reason why they wanted to change the serial number?
Our input data file contains information on mutual fund’s name, scheme code , price , NAV etc. The data input file is separated by character ;. We will use this file to read the data on mutual fund and create another output CSV file that has data of difference between sale price and re-purchase price along with fund’s scheme code and name. In this output file, data in a single line will be separated by character #.
Below is the pom.xml configuration we will need for our Spring batch project.
Context Configuration – job-context.xml
Here we will be having basic configuration of batch job like configuration of Job Repository, transaction manager and other resources details.
We have input file as a CSV file and Spring batch provides an implemented FlatFileItemReader to read these types of files. Though we have the class ready to read CSV file(s), we will still have to provide more configuration on what sort of data is to be read and how we want to convert that into a POJO.
For each line in the CSV file, we will have a POJO created with data in the line. Task of mapping and converting this line to POJO will be handled by lineMapper in FlatFileItemReader. Following is the configuration we will need for reading part of the Spring Batch.
As we see, we have configured the file we want to read as property resource. Attribute linesToSkip tells reader to skip first line while reading the file and start processing data and create POJO from 2nd line onwards. This is done because in our data file, first line contains headers of the data in file. Hence we want to skip that while processing the file. The next property lineMapper has configuration on how we should convert each line in file to POJO object of class MutualFundDataDTO. This is the class we are creating to hold data of file during Spring batch processing.
Reader we are using as above, will process the input file and construct POJO of type MutualFundDataDTO. But to complete thatwe will need to provide information on how each attribute of our POJO will be populated. This is the task of fieldSetMapper inside the lineMapper we have defined.
Now that our reading part is completed, we will go over to the writing part and see how we can create an output file which will have data with different separator of character #. For the writer part, we will be using writer provided by Spring Batch- FlatFileItemWriter. Below is the configuration of the file writer.
Let us look at the configuration we are giving for writer. Property resource gives information on where the output will be written. We can provide the path and file name in this and FlatFileItemWriter will create the file at the location provided and put data in it.
lineAggregator property converts object to a line for the file. We can customize this to suit our needs but for this example we will be using a lineAggregator provided by Spring Batch. DelimitedLineAggregator will convert object to a line in file where data is separated by a fixed character/String.
Note: Using an MSI tuner to create a transform file enables you to customize your supplied Notes installer, however the Tuner lets you access system settings that can wreck havoc if you make errors. Only tune as needed. When in doubt, there is help provided with the InstallShield Tuner for Notes. Installshield tuner for notes. Scroll the list of programs until you find InstallShield Tuner for Lotus Notes or simply click the Search feature and type in 'InstallShield Tuner for Lotus Notes'. If it exists on your system the InstallShield Tuner for Lotus Notes program will be found automatically. Customizing Notes install using the tuner. For related information, see the Lotus Support Site for tech note 1226984 'Is the InstallShield Tuner for Notes available for download.' Related topics. Setting up and customizing Notes installation. Automating Notes installation using a silent install.
fieldExtractor will tell the writer what property of our Java object will be written in to the file. We are again using Spring provided BeanWrapperFieldExtractor and providing names of the property of our POJO-MutualFundDataDTO which will be writen into the file.
Now that we have reader and writer components next step is to create job which we will be running to read and create the files.
Spring Batch execution
To execute the spring batch job we have created we either use Spring provided laucher or create a one for ourselves. Here we will understand of launching the job using our own launcher class. Here we will first load Spring context file we have created above and then get the bean of job and launch it.
When we run this java class in eclipse we can see below output on console
and we can then check the output file created at the location given in configuration. Below is how our project will look like in STS.
Spring batch file job
So we went through the process of using Spring Batch – Reading & Writing file and what sort of configuration and code we will need for this. Do let us your feedback and questions.