0
我寫了一個spring批處理作業，它從db中讀取數據，然後寫入csv。spring批量ItemWriter創建輸出兩次
這項工作非常簡單，它工作，但ItemWriter看起來像它正在執行兩次。即我結束了兩個CSV（完全相同），而不是一個。
有人可以幫我找出原因嗎？它使用不同的jobExecution.id。
附加批處理作業的運行，下面的控制檯輸出...
@Configuration
@EnableBatchProcessing
public class ExportMasterListCsvJobConfig {
@Autowired
public JobBuilderFactory jobBuilderFactory;
@Autowired
public StepBuilderFactory stepBuilderFactory;
@Bean
public Job exportMasterListCsvJob(
Step readStgDbAndExportMasterListStep,
Step writeToCsvStep) {
return jobBuilderFactory.get("exportMasterListCsvJob")
.incrementer(new RunIdIncrementer())
.flow(readStgDbAndExportMasterListStep)
.end()
.build();
}
@Bean
public Step readStgDbAndExportMasterListStep(
@Value("${exportMasterListCsv.generateMasterListRows.chunkSize}") int chunkSize,
@Qualifier("queryOnlineStagingDbReader") ItemReader<MasterList> queryOnlineStagingDbReader,
@Qualifier("masterListFileWriter") ItemWriter<MasterList> masterListFileWriter) {
return stepBuilderFactory.get("readStgDbAndExportMasterListStep")
.<MasterList,MasterList>chunk(chunkSize)
.reader(queryOnlineStagingDbReader)
.writer(masterListFileWriter)
.build();
}
@Bean
public ItemReader<MasterList> queryOnlineStagingDbReader(
DataSource onlineStagingDb,
@Value("${exportMasterListCsv.generateMasterListRows.masterListSql}") String masterListSql) {
JdbcCursorItemReader<MasterList> reader = new JdbcCursorItemReader<>();
reader.setDataSource(onlineStagingDb);
reader.setSql(masterListSql);
reader.setRowMapper(new RowMapper<MasterList>() {
@Override
public MasterList mapRow(ResultSet resultSet, int i) throws SQLException {
MasterList masterList = new MasterList();
masterList.setL2(resultSet.getString("l2"));
masterList.setL2Name(resultSet.getString("l2_name"));
return masterList;
}
});
return reader;
}
@Bean
@StepScope
public ItemWriter<MasterList> masterListFileWriter(
FileSystemResource masterListFile,
@Value("#{stepExecutionContext}")Map<String, Object> executionContext) {
FlatFileItemWriter<MasterList> writer = new FlatFileItemWriter<>();
writer.setResource(masterListFile);
DelimitedLineAggregator<MasterList> lineAggregator = new DelimitedLineAggregator<>();
lineAggregator.setDelimiter(",");
BeanWrapperFieldExtractor<MasterList> extractor = new BeanWrapperFieldExtractor<MasterList>();
extractor.setNames(new String[] { "l2", "l2Name"});
lineAggregator.setFieldExtractor(extractor);
writer.setLineAggregator(lineAggregator);
writer.setForceSync(true);
writer.open(new ExecutionContext(executionContext));
return writer;
}
@Bean
@JobScope
public FileSystemResource masterListFile(
@Value("${temp.dir}") String tempDir,
@Value("#{jobExecution.id}") Long jobId
) throws IOException {
return new FileSystemResource(new File(tempDir, "masterList" + jobId + ".csv"));
}
}
和控制檯輸出...
13:03:45.815 [main] DEBUG org.springframework.test.context.junit4.SpringJUnit4ClassRunner - SpringJUnit4ClassRunner constructor called with [class com.myer.pricing.onlinestore.export.ExportMasterListCsvTest]
13:03:45.827 [main] DEBUG org.springframework.test.context.BootstrapUtils - Instantiating CacheAwareContextLoaderDelegate from class [org.springframework.test.context.cache.DefaultCacheAwareContextLoaderDelegate]
13:03:45.836 [main] DEBUG org.springframework.test.context.BootstrapUtils - Instantiating BootstrapContext using constructor [public org.springframework.test.context.support.DefaultBootstrapContext(java.lang.Class,org.springframework.test.context.CacheAwareContextLoaderDelegate)]
13:03:45.853 [main] DEBUG org.springframework.test.context.BootstrapUtils - Instantiating TestContextBootstrapper for test class [com.myer.pricing.onlinestore.export.ExportMasterListCsvTest] from class [org.springframework.test.context.support.DefaultTestContextBootstrapper]
13:03:45.884 [main] DEBUG org.springframework.test.context.support.DefaultTestContextBootstrapper - Found explicit ContextLoader class [org.springframework.boot.test.SpringApplicationContextLoader] for context configuration attributes [[email protected] declaringClass = 'com.myer.pricing.onlinestore.export.ExportMasterListCsvTest', classes = '{class com.myer.pricing.onlinestore.export.TestJobConfig, class com.myer.pricing.onlinestore.export.job.ExportMasterListCsvJobConfig}', locations = '{}', inheritLocations = true, initializers = '{}', inheritInitializers = true, name = [null], contextLoaderClass = 'org.springframework.boot.test.SpringApplicationContextLoader']
13:03:45.899 [main] DEBUG org.springframework.test.context.support.AbstractContextLoader - Did not detect default resource location for test class [com.myer.pricing.onlinestore.export.ExportMasterListCsvTest]: class path resource [com/myer/pricing/onlinestore/export/ExportMasterListCsvTest-context.xml] does not exist
13:03:45.900 [main] DEBUG org.springframework.test.context.support.AbstractContextLoader - Did not detect default resource location for test class [com.myer.pricing.onlinestore.export.ExportMasterListCsvTest]: class path resource [com/myer/pricing/onlinestore/export/ExportMasterListCsvTestContext.groovy] does not exist
13:03:45.900 [main] INFO org.springframework.test.context.support.AbstractContextLoader - Could not detect default resource locations for test class [com.myer.pricing.onlinestore.export.ExportMasterListCsvTest]: no resource found for suffixes {-context.xml, Context.groovy}.
13:03:45.927 [main] INFO org.springframework.test.context.support.DefaultTestContextBootstrapper - Using TestExecutionListeners: [[email protected]724af044, org.springframework.test[email protected]4678c730, org.springframewor[email protected]6767c1fc, org.springfra[email protected]29ee9faa, org.springframew[email protected]c038203, org.sp[email protected]cc285f4]
13:03:45.931 [main] DEBUG org.springframework.test.annotation.ProfileValueUtils - Retrieved @ProfileValueSourceConfiguration [null] for test class [com.myer.pricing.onlinestore.export.ExportMasterListCsvTest]
13:03:45.931 [main] DEBUG org.springframework.test.annotation.ProfileValueUtils - Retrieved ProfileValueSource type [class org.springframework.test.annotation.SystemProfileValueSource] for class [com.myer.pricing.onlinestore.export.ExportMasterListCsvTest]
13:03:45.946 [main] DEBUG org.springframework.test.annotation.ProfileValueUtils - Retrieved @ProfileValueSourceConfiguration [null] for test class [com.myer.pricing.onlinestore.export.ExportMasterListCsvTest]
13:03:45.946 [main] DEBUG org.springframework.test.annotation.ProfileValueUtils - Retrieved ProfileValueSource type [class org.springframework.test.annotation.SystemProfileValueSource] for class [com.myer.pricing.onlinestore.export.ExportMasterListCsvTest]
13:03:45.948 [main] DEBUG org.springframework.test.annotation.ProfileValueUtils - Retrieved @ProfileValueSourceConfiguration [null] for test class [com.myer.pricing.onlinestore.export.ExportMasterListCsvTest]
13:03:45.948 [main] DEBUG org.springframework.test.annotation.ProfileValueUtils - Retrieved ProfileValueSource type [class org.springframework.test.annotation.SystemProfileValueSource] for class [com.myer.pricing.onlinestore.export.ExportMasterListCsvTest]
13:03:45.950 [main] DEBUG org.springframework.test.annotation.ProfileValueUtils - Retrieved @ProfileValueSourceConfiguration [null] for test class [com.myer.pricing.onlinestore.export.ExportMasterListCsvTest]
13:03:45.950 [main] DEBUG org.springframework.test.annotation.ProfileValueUtils - Retrieved ProfileValueSource type [class org.springframework.test.annotation.SystemProfileValueSource] for class [com.myer.pricing.onlinestore.export.ExportMasterListCsvTest]
13:03:45.953 [main] DEBUG org.springframework.test.context.support.AbstractDirtiesContextTestExecutionListener - Before test class: context [[email protected] testClass = ExportMasterListCsvTest, testInstance = [null], testMethod = [null], testException = [null], mergedContextConfiguration = [[email protected] testClass = ExportMasterListCsvTest, locations = '{}', classes = '{class com.myer.pricing.onlinestore.export.TestJobConfig, class com.myer.pricing.onlinestore.export.job.ExportMasterListCsvJobConfig}', contextInitializerClasses = '[]', activeProfiles = '{batchtest}', propertySourceLocations = '{}', propertySourceProperties = '{}', contextLoader = 'org.springframework.boot.test.SpringApplicationContextLoader', parent = [null]]], class annotated with @DirtiesContext [false] with mode [null].
13:03:45.954 [main] DEBUG org.springframework.test.annotation.ProfileValueUtils - Retrieved @ProfileValueSourceConfiguration [null] for test class [com.myer.pricing.onlinestore.export.ExportMasterListCsvTest]
13:03:45.954 [main] DEBUG org.springframework.test.annotation.ProfileValueUtils - Retrieved ProfileValueSource type [class org.springframework.test.annotation.SystemProfileValueSource] for class [com.myer.pricing.onlinestore.export.ExportMasterListCsvTest]
13:03:45.959 [main] DEBUG org.springframework.test.util.ReflectionTestUtils - Getting field 'mergedContextConfiguration' from target object [[[email protected] testClass = ExportMasterListCsvTest, testInstance = [email protected]2d7, testMethod = [null], testException = [null], mergedContextConfiguration = [[email protected] testClass = ExportMasterListCsvTest, locations = '{}', classes = '{class com.myer.pricing.onlinestore.export.TestJobConfig, class com.myer.pricing.onlinestore.export.job.ExportMasterListCsvJobConfig}', contextInitializerClasses = '[]', activeProfiles = '{batchtest}', propertySourceLocations = '{}', propertySourceProperties = '{}', contextLoader = 'org.springframework.boot.test.SpringApplicationContextLoader', parent = [null]]]] or target class [class org.springframework.test.context.support.DefaultTestContext]
13:03:45.960 [main] DEBUG org.springframework.test.util.ReflectionTestUtils - Setting field 'propertySourceProperties' of type [null] on target object [[[email protected] testClass = ExportMasterListCsvTest, locations = '{}', classes = '{class com.myer.pricing.onlinestore.export.TestJobConfig, class com.myer.pricing.onlinestore.export.job.ExportMasterListCsvJobConfig}', contextInitializerClasses = '[]', activeProfiles = '{batchtest}', propertySourceLocations = '{}', propertySourceProperties = '{}', contextLoader = 'org.springframework.boot.test.SpringApplicationContextLoader', parent = [null]]] or target class [class org.springframework.test.context.MergedContextConfiguration] to value [[Ljava.lang.String;@4667ae56]
13:03:45.961 [main] DEBUG org.springframework.test.context.support.DependencyInjectionTestExecutionListener - Performing dependency injection for test context [[[email protected] testClass = ExportMasterListCsvTest, testInstance = [email protected]2d7, testMethod = [null], testException = [null], mergedContextConfiguration = [[email protected] testClass = ExportMasterListCsvTest, locations = '{}', classes = '{class com.myer.pricing.onlinestore.export.TestJobConfig, class com.myer.pricing.onlinestore.export.job.ExportMasterListCsvJobConfig}', contextInitializerClasses = '[]', activeProfiles = '{batchtest}', propertySourceLocations = '{}', propertySourceProperties = '{org.springframework.boot.test.IntegrationTest=true}', contextLoader = 'org.springframework.boot.test.SpringApplicationContextLoader', parent = [null]]]].
13:03:46.072 [main] DEBUG org.springframework.core.env.StandardEnvironment - Adding [systemProperties] PropertySource with lowest search precedence
13:03:46.073 [main] DEBUG org.springframework.core.env.StandardEnvironment - Adding [systemEnvironment] PropertySource with lowest search precedence
13:03:46.074 [main] DEBUG org.springframework.core.env.StandardEnvironment - Initialized StandardEnvironment with PropertySources [systemProperties,systemEnvironment]
13:03:46.074 [main] DEBUG org.springframework.core.env.StandardEnvironment - Adding [test] PropertySource with highest search precedence
13:03:46.075 [main] DEBUG org.springframework.core.env.StandardEnvironment - Adding [integrationTest] PropertySource with search precedence immediately lower than [systemEnvironment]
. ____ _ __ _ _
/\\/___'_ __ _ _(_)_ __ __ _ \ \ \ \
(()\___ | '_ | '_| | '_ \/ _` | \ \ \ \
\\/ ___)| |_)| | | | | || (_| | ))))
' |____| .__|_| |_|_| |_\__, |////
=========|_|==============|___/=/_/_/_/
:: Spring Boot :: (v1.3.6.RELEASE)
2016-07-22 13:03:46.541 INFO 10084 --- [ main] c.m.p.o.export.ExportMasterListCsvTest : Starting ExportMasterListCsvTest on IGSM473537 with PID 10084 (C:\design_centre_dev\myer-pricing-engine\java-onlinestore-feed-exporter\target\test-classes started by rriviere in C:\design_centre_dev\myer-pricing-engine\java-onlinestore-feed-exporter)
2016-07-22 13:03:46.542 INFO 10084 --- [ main] c.m.p.o.export.ExportMasterListCsvTest : The following profiles are active: batchtest
2016-07-22 13:03:46.875 INFO 10084 --- [ main] s.c.a.AnnotationConfigApplicationContext : Refreshing org.spring[email protected]4e08711f: startup date [Fri Jul 22 13:03:46 AEST 2016]; root of context hierarchy
2016-07-22 13:03:48.151 INFO 10084 --- [ main] o.s.b.f.config.PropertiesFactoryBean : Loading properties file from URL [jar:file:/C:/.m2/repository/org/springframework/integration/spring-integration-core/4.2.8.RELEASE/spring-integration-core-4.2.8.RELEASE.jar!/META-INF/spring.integration.default.properties]
2016-07-22 13:03:48.157 INFO 10084 --- [ main] o.s.i.config.IntegrationRegistrar : No bean named 'integrationHeaderChannelRegistry' has been explicitly defined. Therefore, a default DefaultHeaderChannelRegistry will be created.
2016-07-22 13:03:48.716 WARN 10084 --- [ main] o.s.c.a.ConfigurationClassEnhancer : @Bean method ScopeConfiguration.stepScope is non-static and returns an object assignable to Spring's BeanFactoryPostProcessor interface. This will result in a failure to process annotations such as @Autowired, @Resource and @PostConstruct within the method's declaring @Configuration class. Add the 'static' modifier to this method to avoid these container lifecycle issues; see @Bean javadoc for complete details.
2016-07-22 13:03:48.728 WARN 10084 --- [ main] o.s.c.a.ConfigurationClassEnhancer : @Bean method ScopeConfiguration.jobScope is non-static and returns an object assignable to Spring's BeanFactoryPostProcessor interface. This will result in a failure to process annotations such as @Autowired, @Resource and @PostConstruct within the method's declaring @Configuration class. Add the 'static' modifier to this method to avoid these container lifecycle issues; see @Bean javadoc for complete details.
2016-07-22 13:03:48.742 INFO 10084 --- [ main] faultConfiguringBeanFactoryPostProcessor : No bean named 'errorChannel' has been explicitly defined. Therefore, a default PublishSubscribeChannel will be created.
2016-07-22 13:03:48.751 INFO 10084 --- [ main] faultConfiguringBeanFactoryPostProcessor : No bean named 'taskScheduler' has been explicitly defined. Therefore, a default ThreadPoolTaskScheduler will be created.
2016-07-22 13:03:48.900 INFO 10084 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'org.springframework.transaction.annotation.ProxyTransactionManagementConfiguration' of type [class org.springframework.transaction.annotation.ProxyTransactionManagementConfiguration$$EnhancerBySpringCGLIB$$402d705e] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2016-07-22 13:03:49.060 INFO 10084 --- [ main] o.s.j.d.e.EmbeddedDatabaseFactory : Starting embedded database: url='jdbc:hsqldb:mem:testdb', username='sa'
2016-07-22 13:03:49.459 INFO 10084 --- [ main] o.s.jdbc.datasource.init.ScriptUtils : Executing SQL script from class path resource [db/sql/staging_db.sql]
2016-07-22 13:03:49.465 INFO 10084 --- [ main] o.s.jdbc.datasource.init.ScriptUtils : Executed SQL script from class path resource [db/sql/staging_db.sql] in 6 ms.
2016-07-22 13:03:49.710 WARN 10084 --- [ main] o.s.b.c.l.AbstractListenerFactoryBean : org.springframework.batch.item.ItemWriter is an interface. The implementing class will not be queried for annotation based listener configurations. If using @StepScope on a @Bean method, be sure to return the implementing class so listner annotations can be used.
2016-07-22 13:03:49.797 INFO 10084 --- [ main] o.s.aop.framework.CglibAopProxy : Unable to proxy method [public final java.lang.String org.springframework.core.io.FileSystemResource.getPath()] because it is final: All calls to this method via a proxy will NOT be routed to the target instance.
2016-07-22 13:03:50.236 INFO 10084 --- [ main] o.s.b.f.config.PropertiesFactoryBean : Loading properties file from URL [jar:file:/C:/.m2/repository/org/springframework/integration/spring-integration-core/4.2.8.RELEASE/spring-integration-core-4.2.8.RELEASE.jar!/META-INF/spring.integration.default.properties]
2016-07-22 13:03:50.730 INFO 10084 --- [ main] o.s.s.c.ThreadPoolTaskScheduler : Initializing ExecutorService 'taskScheduler'
2016-07-22 13:03:51.258 INFO 10084 --- [ main] o.s.jdbc.datasource.init.ScriptUtils : Executing SQL script from class path resource [org/springframework/batch/core/schema-hsqldb.sql]
2016-07-22 13:03:51.281 INFO 10084 --- [ main] o.s.jdbc.datasource.init.ScriptUtils : Executed SQL script from class path resource [org/springframework/batch/core/schema-hsqldb.sql] in 23 ms.
2016-07-22 13:03:51.813 INFO 10084 --- [ main] o.s.c.support.DefaultLifecycleProcessor : Starting beans in phase 0
2016-07-22 13:03:51.814 INFO 10084 --- [ main] o.s.i.endpoint.EventDrivenConsumer : Adding {logging-channel-adapter:_org.springframework.integration.errorLogger} as a subscriber to the 'errorChannel' channel
2016-07-22 13:03:51.814 INFO 10084 --- [ main] o.s.i.channel.PublishSubscribeChannel : Channel 'application:batchtest.errorChannel' has 1 subscriber(s).
2016-07-22 13:03:51.814 INFO 10084 --- [ main] o.s.i.endpoint.EventDrivenConsumer : started _org.springframework.integration.errorLogger
2016-07-22 13:03:51.832 INFO 10084 --- [ main] o.s.b.a.b.JobLauncherCommandLineRunner : Running default command line with: []
2016-07-22 13:03:51.844 INFO 10084 --- [ main] o.s.b.c.r.s.JobRepositoryFactoryBean : No database type set, using meta data indicating: HSQL
2016-07-22 13:03:51.973 INFO 10084 --- [ main] o.s.b.c.l.support.SimpleJobLauncher : No TaskExecutor has been set, defaulting to synchronous executor.
2016-07-22 13:03:52.044 INFO 10084 --- [ main] o.s.b.c.l.support.SimpleJobLauncher : Job: [FlowJob: [name=exportMasterListCsvJob]] launched with the following parameters: [{run.id=1}]
2016-07-22 13:03:52.073 INFO 10084 --- [ main] o.s.batch.core.job.SimpleStepHandler : Executing step: [readStgDbAndExportMasterListStep]
2016-07-22 13:03:52.167 INFO 10084 --- [ main] o.s.b.c.l.support.SimpleJobLauncher : Job: [FlowJob: [name=exportMasterListCsvJob]] completed with the following parameters: [{run.id=1}] and the following status: [COMPLETED]
2016-07-22 13:03:52.169 INFO 10084 --- [ main] c.m.p.o.export.ExportMasterListCsvTest : Started ExportMasterListCsvTest in 6.091 seconds (JVM running for 6.895)
2016-07-22 13:03:52.196 INFO 10084 --- [ main] o.s.b.c.l.support.SimpleJobLauncher : Job: [FlowJob: [name=exportMasterListCsvJob]] launched with the following parameters: [{random=93853}]
2016-07-22 13:03:52.234 INFO 10084 --- [ main] o.s.batch.core.job.SimpleStepHandler : Executing step: [readStgDbAndExportMasterListStep]
2016-07-22 13:03:52.290 INFO 10084 --- [ main] o.s.b.c.l.support.SimpleJobLauncher : Job: [FlowJob: [name=exportMasterListCsvJob]] completed with the following parameters: [{random=93853}] and the following status: [COMPLETED]
2016-07-22 13:03:52.305 INFO 10084 --- [ Thread-1] s.c.a.AnnotationConfigApplicationContext : Closing org.spring[email protected]4e08711f: startup date [Fri Jul 22 13:03:46 AEST 2016]; root of context hierarchy
2016-07-22 13:03:52.307 INFO 10084 --- [ Thread-1] o.s.c.support.DefaultLifecycleProcessor : Stopping beans in phase 0
2016-07-22 13:03:52.308 INFO 10084 --- [ Thread-1] o.s.i.endpoint.EventDrivenConsumer : Removing {logging-channel-adapter:_org.springframework.integration.errorLogger} as a subscriber to the 'errorChannel' channel
2016-07-22 13:03:52.309 INFO 10084 --- [ Thread-1] o.s.i.channel.PublishSubscribeChannel : Channel 'application:batchtest.errorChannel' has 0 subscriber(s).
2016-07-22 13:03:52.309 INFO 10084 --- [ Thread-1] o.s.i.endpoint.EventDrivenConsumer : stopped _org.springframework.integration.errorLogger
2016-07-22 13:03:52.314 INFO 10084 --- [ Thread-1] o.s.s.c.ThreadPoolTaskScheduler : Shutting down ExecutorService 'taskScheduler'
2016-07-22 13:03:52.317 INFO 10084 --- [ Thread-1] o.s.j.d.e.EmbeddedDatabaseFactory : Shutting down embedded database: url='jdbc:hsqldb:mem:testdb'
這產生了兩個CSV的每一個不同的jobExecution.id我已附加到文件名稱。 masterList0.csv和masterList1.csv
在此先感謝