[SpringBatch 실습] 30. 스프링 배치 MultiThread(멀티 스레드) Job 수행하여 분석

shbada
Coding/Spring Batch
2022. 7. 7.

728x90

스프링배치 Multi-thread

Step 내에서 멀티 스레드로 Chunk 기반 처리가 이루어지는 구조다.
TaskExecutorRepeatTemplate이 반복자로 사용된다.
설정한 개수만큼의 스레드를 생성하여 수행한다.
ItemReader는 Thread-safe 인지 반드시 확인해야 한다.
- 데이터를 소스로부터 읽어오는 역할이기 때문에 스레드마다 중복해서 데이터를 읽어오지 않도록 동기화가 보장되어야한다.
스레드마다 새로운 Chunk가 할당되어 데이터 동기화가 보장된다.
- 스레드끼리 Chunk를 서로 공유하지 않는다.

예제코드

MultiThreadCustomer.java

package com.project.springbatch._43_multiThread;

import lombok.AllArgsConstructor;
import lombok.Data;

import java.util.Date;

@Data
@AllArgsConstructor
public class MultiThreadCustomer {
    // private String name;
    private final long id;
    private final String firstName;
    private final String lastName;
    private final Date birthdate;
}

MultiThreadCustomerRowMapper.java

package com.project.springbatch._43_multiThread;

import org.springframework.jdbc.core.RowMapper;

import java.sql.ResultSet;
import java.sql.SQLException;

public class MultiThreadCustomerRowMapper implements RowMapper<MultiThreadCustomer> {
    @Override
    public MultiThreadCustomer mapRow(ResultSet rs, int i) throws SQLException {
        return new MultiThreadCustomer(rs.getLong("id"),
                rs.getString("firstName"),
                rs.getString("lastName"),
                rs.getDate("birthdate"));
    }
}

Listener 구성

MultiThreadCustomReadListener.java

package com.project.springbatch._43_multiThread;

import org.springframework.batch.core.ItemReadListener;

public class MultiThreadCustomReadListener implements ItemReadListener<MultiThreadCustomer> {

    @Override
    public void beforeRead() {

    }

    @Override
    public void afterRead(MultiThreadCustomer item) {
        System.out.println("Thread : " + Thread.currentThread().getName() + ", read item : " + item.getId());
    }

    @Override
    public void onReadError(Exception ex) {

    }
}

MultiThreadCustomProcessListener.java

package com.project.springbatch._43_multiThread;

import org.springframework.batch.core.ItemProcessListener;

public class MultiThreadCustomProcessListener implements ItemProcessListener<MultiThreadCustomer, MultiThreadCustomer> {
    @Override
    public void beforeProcess(MultiThreadCustomer item) {

    }

    @Override
    public void afterProcess(MultiThreadCustomer item, MultiThreadCustomer result) {
        System.out.println("Thread : " + Thread.currentThread().getName() + ", process item : " + item.getId());
    }

    @Override
    public void onProcessError(MultiThreadCustomer item, Exception e) {

    }
}

MultiThreadCustomWriterListener.java

package com.project.springbatch._43_multiThread;

import org.springframework.batch.core.ItemWriteListener;

import java.util.List;

public class MultiThreadCustomWriteListener implements ItemWriteListener<MultiThreadCustomer> {

    @Override
    public void beforeWrite(List<? extends MultiThreadCustomer> items) {

    }

    @Override
    public void afterWrite(List<? extends MultiThreadCustomer> items) {
        System.out.println("Thread : " + Thread.currentThread().getName() + ", write items : " + items.size());

    }

    @Override
    public void onWriteError(Exception exception, List<? extends MultiThreadCustomer> items) {

    }
}

MultiThreadStopWatchJobListener.java

package com.project.springbatch._43_multiThread;

import org.springframework.batch.core.JobExecution;
import org.springframework.batch.core.JobExecutionListener;

public class MultiThreadStopWatchJobListener implements JobExecutionListener {
    @Override
    public void beforeJob(JobExecution jobExecution) {

    }

    @Override
    public void afterJob(JobExecution jobExecution) {
        long time = jobExecution.getEndTime().getTime() - jobExecution.getStartTime().getTime();
        System.out.println("time : " + time);
    }
}

Job 생성

package com.project.springbatch._43_multiThread;

import lombok.RequiredArgsConstructor;
import org.springframework.batch.core.Job;
import org.springframework.batch.core.Step;
import org.springframework.batch.core.configuration.annotation.JobBuilderFactory;
import org.springframework.batch.core.configuration.annotation.StepBuilderFactory;
import org.springframework.batch.core.launch.support.RunIdIncrementer;
import org.springframework.batch.item.ItemProcessor;
import org.springframework.batch.item.database.*;
import org.springframework.batch.item.database.builder.JdbcCursorItemReaderBuilder;
import org.springframework.batch.item.database.support.MySqlPagingQueryProvider;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.core.task.TaskExecutor;
import org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor;

import javax.sql.DataSource;
import java.util.HashMap;
import java.util.Map;

/*
--job.name=multiThreadJob
*/
@RequiredArgsConstructor
@Configuration
public class MultiThreadStepConfiguration {

    private final JobBuilderFactory jobBuilderFactory;
    private final StepBuilderFactory stepBuilderFactory;
    private final DataSource dataSource;

    @Bean
    public Job multiThreadJob() throws Exception {
        return jobBuilderFactory.get("multiThreadJob")
                .incrementer(new RunIdIncrementer())
                .start(multiThreadStep1())
                .listener(new MultiThreadStopWatchJobListener())
                .build();
    }

    @Bean
    public Step multiThreadStep1() throws Exception {
        return stepBuilderFactory.get("multiThreadStep1")
                .<MultiThreadCustomer, MultiThreadCustomer>chunk(100)
                .reader(multiThreadCustomPagingItemReader())
                .listener(new MultiThreadCustomReadListener())
                .processor((ItemProcessor<MultiThreadCustomer, MultiThreadCustomer>) item -> item)
                .listener(new MultiThreadCustomProcessListener())
                .writer(multiThreadCustomItemWriter())
                .listener(new MultiThreadCustomWriteListener())
                .taskExecutor(multiThreadTaskExecutor())
//                .throttleLimit(2)
                .build();
    }

    @Bean
    public JdbcPagingItemReader<MultiThreadCustomer> multiThreadCustomPagingItemReader() {
        JdbcPagingItemReader<MultiThreadCustomer> reader = new JdbcPagingItemReader<>();

        reader.setDataSource(this.dataSource);
        reader.setPageSize(100);
        reader.setRowMapper(new MultiThreadCustomerRowMapper());

        MySqlPagingQueryProvider queryProvider = new MySqlPagingQueryProvider();
        queryProvider.setSelectClause("id, firstName, lastName, birthdate");
        queryProvider.setFromClause("from customer");

        Map<String, Order> sortKeys = new HashMap<>(1);

        sortKeys.put("id", Order.ASCENDING);

        queryProvider.setSortKeys(sortKeys);

        reader.setQueryProvider(queryProvider);

        return reader;
    }

    @Bean
    public JdbcBatchItemWriter multiThreadCustomItemWriter() {
        JdbcBatchItemWriter<MultiThreadCustomer> itemWriter = new JdbcBatchItemWriter<>();

        itemWriter.setDataSource(this.dataSource);
        itemWriter.setSql("insert into customer2 values (:id, :firstName, :lastName, :birthdate)");
        itemWriter.setItemSqlParameterSourceProvider(new BeanPropertyItemSqlParameterSourceProvider());
        itemWriter.afterPropertiesSet();

        return itemWriter;
    }

    @Bean
    public TaskExecutor multiThreadTaskExecutor(){
        ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
        executor.setCorePoolSize(4); // 몇개의 스레드를 관리할것인지?
        executor.setMaxPoolSize(8); // 4개의 스레드가 작업을 처리하고 있는데, 나머지 처리되지 않은 task가 있을때 얼마만큼의 최대 스레드를 생성할것인지?
        executor.setThreadNamePrefix("async-thread-");
        return executor;
    }
}

1) 리스너 설정

@Bean
public Step multiThreadStep1() throws Exception {
    return stepBuilderFactory.get("multiThreadStep1")
            .<MultiThreadCustomer, MultiThreadCustomer>chunk(100)
            .reader(multiThreadCustomPagingItemReader())
            .listener(new MultiThreadCustomReadListener())
            .processor((ItemProcessor<MultiThreadCustomer, MultiThreadCustomer>) item -> item)
            .listener(new MultiThreadCustomProcessListener())
            .writer(multiThreadCustomItemWriter())
            .listener(new MultiThreadCustomWriteListener())
            ...
            .build();
}

2) taskExecutor() 설정

.taskExecutor(multiThreadTaskExecutor())

4개의 스레드로 지정

@Bean
public TaskExecutor multiThreadTaskExecutor(){
    ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
    executor.setCorePoolSize(4); // 몇개의 스레드를 관리할것인지?
    executor.setMaxPoolSize(8); // 4개의 스레드가 작업을 처리하고 있는데, 나머지 처리되지 않은 task가 있을때 얼마만큼의 최대 스레드를 생성할것인지?
    executor.setThreadNamePrefix("async-thread-");
    return executor;
}

수행결과

총 4개의 스레드가 동시에 수행하고, 각각의 스레드는 chunk size만큼 수행한다.

async-thread-1
async-thread-2
async-thread-3
async-thread-4

Thread : async-thread-1, read item : 3
Thread : async-thread-4, read item : 4
Thread : async-thread-3, read item : 2
Thread : async-thread-2, read item : 1
Thread : async-thread-4, read item : 5
Thread : async-thread-3, read item : 6
Thread : async-thread-2, read item : 8
Thread : async-thread-4, read item : 9
Thread : async-thread-1, read item : 7
Thread : async-thread-4, read item : 12
Thread : async-thread-2, read item : 11
Thread : async-thread-1, read item : 13
Thread : async-thread-3, read item : 10
Thread : async-thread-2, read item : 15
Thread : async-thread-1, read item : 16
Thread : async-thread-3, read item : 17
Thread : async-thread-2, read item : 18
Thread : async-thread-4, read item : 14
Thread : async-thread-1, read item : 19
Thread : async-thread-2, read item : 21
Thread : async-thread-4, read item : 22
Thread : async-thread-1, read item : 23
Thread : async-thread-2, read item : 24
Thread : async-thread-4, read item : 25
Thread : async-thread-3, read item : 20
Thread : async-thread-1, read item : 26
Thread : async-thread-4, read item : 28
Thread : async-thread-3, read item : 29
Thread : async-thread-2, read item : 27
Thread : async-thread-4, read item : 31
Thread : async-thread-3, read item : 32
Thread : async-thread-1, read item : 30
Thread : async-thread-2, read item : 33
Thread : async-thread-3, read item : 35
Thread : async-thread-1, read item : 36
Thread : async-thread-4, read item : 34
Thread : async-thread-3, read item : 38
Thread : async-thread-2, read item : 37
Thread : async-thread-1, read item : 39
Thread : async-thread-3, read item : 40
Thread : async-thread-4, read item : 41
Thread : async-thread-1, read item : 43
Thread : async-thread-2, read item : 42
Thread : async-thread-4, read item : 45
Thread : async-thread-1, read item : 46
Thread : async-thread-3, read item : 44
Thread : async-thread-2, read item : 47
Thread : async-thread-1, read item : 49
Thread : async-thread-3, read item : 50
Thread : async-thread-2, read item : 51
Thread : async-thread-1, read item : 52
Thread : async-thread-3, read item : 53
Thread : async-thread-2, read item : 54
Thread : async-thread-1, read item : 55
Thread : async-thread-4, read item : 48
Thread : async-thread-1, read item : 58
Thread : async-thread-4, read item : 59
Thread : async-thread-1, read item : 60
Thread : async-thread-2, read item : 57
Thread : async-thread-3, read item : 56
Thread : async-thread-2, read item : 63
Thread : async-thread-1, read item : 62
Thread : async-thread-2, read item : 65
Thread : async-thread-4, read item : 61
Thread : async-thread-1, read item : 66
Thread : async-thread-3, read item : 64
Thread : async-thread-1, read item : 68
Thread : async-thread-2, read item : 67
Thread : async-thread-1, read item : 71
Thread : async-thread-3, read item : 70
Thread : async-thread-4, read item : 69
Thread : async-thread-2, read item : 73
Thread : async-thread-1, read item : 72
Thread : async-thread-3, read item : 74
Thread : async-thread-4, read item : 75
Thread : async-thread-2, read item : 76
Thread : async-thread-1, read item : 78
Thread : async-thread-3, read item : 77
Thread : async-thread-4, read item : 79
Thread : async-thread-1, read item : 81
Thread : async-thread-3, read item : 82
Thread : async-thread-2, read item : 80
Thread : async-thread-4, read item : 84
Thread : async-thread-1, read item : 83
Thread : async-thread-2, read item : 86
Thread : async-thread-3, read item : 85
Thread : async-thread-1, read item : 88
Thread : async-thread-4, read item : 87
Thread : async-thread-2, read item : 89
Thread : async-thread-3, read item : 90
Thread : async-thread-1, read item : 91
Thread : async-thread-2, read item : 93
Thread : async-thread-4, read item : 92
Thread : async-thread-1, read item : 95
Thread : async-thread-2, read item : 96
Thread : async-thread-4, read item : 97
Thread : async-thread-3, read item : 94
Thread : async-thread-4, read item : 100
Thread : async-thread-2, read item : 99
Thread : async-thread-1, read item : 98
Thread : async-thread-3, process item : 2
Thread : async-thread-4, process item : 4
Thread : async-thread-2, process item : 1
Thread : async-thread-1, process item : 3
Thread : async-thread-3, process item : 6
Thread : async-thread-1, process item : 7
Thread : async-thread-2, process item : 8
Thread : async-thread-4, process item : 5
Thread : async-thread-1, process item : 13
Thread : async-thread-3, process item : 10
Thread : async-thread-2, process item : 11
Thread : async-thread-1, process item : 16
Thread : async-thread-4, process item : 9
Thread : async-thread-3, process item : 17
Thread : async-thread-1, process item : 19
Thread : async-thread-2, process item : 15
Thread : async-thread-4, process item : 12
Thread : async-thread-1, process item : 23
Thread : async-thread-2, process item : 18
Thread : async-thread-3, process item : 20
Thread : async-thread-4, process item : 14
Thread : async-thread-2, process item : 21
Thread : async-thread-1, process item : 26
Thread : async-thread-4, process item : 22
Thread : async-thread-3, process item : 29
Thread : async-thread-1, process item : 30
Thread : async-thread-2, process item : 24
Thread : async-thread-3, process item : 32
Thread : async-thread-4, process item : 25
Thread : async-thread-2, process item : 27
Thread : async-thread-1, process item : 36
Thread : async-thread-3, process item : 35
Thread : async-thread-2, process item : 33
Thread : async-thread-4, process item : 28
Thread : async-thread-1, process item : 39
Thread : async-thread-2, process item : 37
Thread : async-thread-3, process item : 38
Thread : async-thread-1, process item : 43
Thread : async-thread-4, process item : 31
Thread : async-thread-3, process item : 40
Thread : async-thread-1, process item : 46
Thread : async-thread-4, process item : 34
Thread : async-thread-2, process item : 42
Thread : async-thread-3, process item : 44
Thread : async-thread-4, process item : 41
Thread : async-thread-2, process item : 47
Thread : async-thread-1, process item : 49
Thread : async-thread-3, process item : 50
Thread : async-thread-4, process item : 45
Thread : async-thread-2, process item : 51
Thread : async-thread-1, process item : 52
Thread : async-thread-4, process item : 48
Thread : async-thread-3, process item : 53
Thread : async-thread-1, process item : 55
Thread : async-thread-4, process item : 59
Thread : async-thread-3, process item : 56
Thread : async-thread-2, process item : 54
Thread : async-thread-1, process item : 58
Thread : async-thread-3, process item : 64
Thread : async-thread-4, process item : 61
Thread : async-thread-1, process item : 60
Thread : async-thread-2, process item : 57
Thread : async-thread-4, process item : 69
Thread : async-thread-3, process item : 70
Thread : async-thread-1, process item : 62
Thread : async-thread-2, process item : 63
Thread : async-thread-4, process item : 75
Thread : async-thread-3, process item : 74
Thread : async-thread-1, process item : 66
Thread : async-thread-2, process item : 65
Thread : async-thread-4, process item : 79
Thread : async-thread-3, process item : 77
Thread : async-thread-2, process item : 67
Thread : async-thread-1, process item : 68
Thread : async-thread-2, process item : 73
Thread : async-thread-3, process item : 82
Thread : async-thread-1, process item : 71
Thread : async-thread-4, process item : 84
Thread : async-thread-3, process item : 85
Thread : async-thread-1, process item : 72
Thread : async-thread-2, process item : 76
Thread : async-thread-4, process item : 87
Thread : async-thread-3, process item : 90
Thread : async-thread-2, process item : 80
Thread : async-thread-1, process item : 78
Thread : async-thread-4, process item : 92
Thread : async-thread-2, process item : 86
Thread : async-thread-3, process item : 94
Thread : async-thread-4, process item : 97
Thread : async-thread-2, process item : 89
Thread : async-thread-1, process item : 81
Thread : async-thread-4, process item : 100
Thread : async-thread-2, process item : 93
Thread : async-thread-1, process item : 83
Thread : async-thread-2, process item : 96
Thread : async-thread-1, process item : 88
Thread : async-thread-2, process item : 99
Thread : async-thread-1, process item : 91
Thread : async-thread-1, process item : 95
Thread : async-thread-1, process item : 98
Thread : async-thread-2, write items : 26
Thread : async-thread-3, write items : 22
Thread : async-thread-4, write items : 23
Thread : async-thread-1, write items : 29

디버깅

1) RepeatTemplate.java > executeInternal()

try {

   while (running) {

      /*
       * Run the before interceptors here, not in the task executor so
       * that they all happen in the same thread - it's easier for
       * tracking batch status, amongst other things.
       */
      for (int i = 0; i < listeners.length; i++) {
         RepeatListener interceptor = listeners[i];
         interceptor.before(context);
         // Allow before interceptors to veto the batch by setting
         // flag.
         running = running && !isMarkedComplete(context);
      }

      // Check that we are still running (should always be true) ...
      if (running) {

         try {

            result = getNextResult(context, callback, state);
            executeAfterInterceptors(context, result);

         }
         catch (Throwable throwable) {
            doHandle(throwable, context, deferred);
         }

         // N.B. the order may be important here:
         if (isComplete(context, result) || isMarkedComplete(context) || !deferred.isEmpty()) {
            running = false;
         }

      }

   }

   result = result.and(waitForResults(state));
   for (Throwable throwable : throwables) {
      doHandle(throwable, context, deferred);
   }

   // Explicitly drop any references to internal state...
   state = null;

}

2) TaskExecutorRepeatTemplate.java > getNextResult()

// 조절 제한 개수
private int throttleLimit = DEFAULT_THROTTLE_LIMIT;

// Thread를 조절 제한 수 만큼 생성하고 Task를 할당 
private TaskExecutor taskExecutor = new SyncTaskExecutor();

참고로 throttleLimit은 아래와 같이 우리가 생성한 Step() 안에서 설정할 수도 있다.

@Bean
public Step multiThreadStep1() throws Exception {
    return stepBuilderFactory.get("multiThreadStep1")
            ...
            .taskExecutor(multiThreadTaskExecutor())
            .throttleLimit(2)
            .build();
}

3) TaskExecutorRepeatTemplate.java > ExecutionRunnable > run()

// 반복기에 의해 호출되는 콜백
private final RepeatCallback callback;

// 최종 repeatStatus를 담고있는 Queue
private final ResultQueue<ResultHolder> queue;

// RepeatCallback 로직 수행
public void run() {...}

throttleLimit() 설정해보자.

package com.project.springbatch._43_multiThread;

import lombok.RequiredArgsConstructor;
import org.springframework.batch.core.Job;
import org.springframework.batch.core.Step;
import org.springframework.batch.core.configuration.annotation.JobBuilderFactory;
import org.springframework.batch.core.configuration.annotation.StepBuilderFactory;
import org.springframework.batch.core.launch.support.RunIdIncrementer;
import org.springframework.batch.item.ItemProcessor;
import org.springframework.batch.item.database.*;
import org.springframework.batch.item.database.builder.JdbcCursorItemReaderBuilder;
import org.springframework.batch.item.database.support.MySqlPagingQueryProvider;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.core.task.TaskExecutor;
import org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor;

import javax.sql.DataSource;
import java.util.HashMap;
import java.util.Map;

/*
--job.name=multiThreadJob
*/
@RequiredArgsConstructor
@Configuration
public class MultiThreadStepConfiguration {

    private final JobBuilderFactory jobBuilderFactory;
    private final StepBuilderFactory stepBuilderFactory;
    private final DataSource dataSource;

    @Bean
    public Job multiThreadJob() throws Exception {
        return jobBuilderFactory.get("multiThreadJob")
                .incrementer(new RunIdIncrementer())
                .start(multiThreadStep1())
                .listener(new MultiThreadStopWatchJobListener())
                .build();
    }

    @Bean
    public Step multiThreadStep1() throws Exception {
        return stepBuilderFactory.get("multiThreadStep1")
                .<MultiThreadCustomer, MultiThreadCustomer>chunk(100)
                .reader(multiThreadCustomPagingItemReader())
                .listener(new MultiThreadCustomReadListener())
                .processor((ItemProcessor<MultiThreadCustomer, MultiThreadCustomer>) item -> item)
                .listener(new MultiThreadCustomProcessListener())
                .writer(multiThreadCustomItemWriter())
                .listener(new MultiThreadCustomWriteListener())
                .taskExecutor(multiThreadTaskExecutor())
                .throttleLimit(2)
                .build();
    }

    @Bean
    public JdbcPagingItemReader<MultiThreadCustomer> multiThreadCustomPagingItemReader() {
        JdbcPagingItemReader<MultiThreadCustomer> reader = new JdbcPagingItemReader<>();

        reader.setDataSource(this.dataSource);
        reader.setPageSize(100);
        reader.setRowMapper(new MultiThreadCustomerRowMapper());

        MySqlPagingQueryProvider queryProvider = new MySqlPagingQueryProvider();
        queryProvider.setSelectClause("id, firstName, lastName, birthdate");
        queryProvider.setFromClause("from customer");

        Map<String, Order> sortKeys = new HashMap<>(1);

        sortKeys.put("id", Order.ASCENDING);

        queryProvider.setSortKeys(sortKeys);

        reader.setQueryProvider(queryProvider);

        return reader;
    }

    @Bean
    public JdbcBatchItemWriter multiThreadCustomItemWriter() {
        JdbcBatchItemWriter<MultiThreadCustomer> itemWriter = new JdbcBatchItemWriter<>();

        itemWriter.setDataSource(this.dataSource);
        itemWriter.setSql("insert into customer2 values (:id, :firstName, :lastName, :birthdate)");
        itemWriter.setItemSqlParameterSourceProvider(new BeanPropertyItemSqlParameterSourceProvider());
        itemWriter.afterPropertiesSet();

        return itemWriter;
    }

    @Bean
    public TaskExecutor multiThreadTaskExecutor(){
        ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
        executor.setCorePoolSize(4); // 몇개의 스레드를 관리할것인지?
        executor.setMaxPoolSize(8); // 4개의 스레드가 작업을 처리하고 있는데, 나머지 처리되지 않은 task가 있을때 얼마만큼의 최대 스레드를 생성할것인지?
        executor.setThreadNamePrefix("async-thread-");
        return executor;
    }
}

1) .throttleLimit(2)

throttleLimit(2)로 설정하여 2개만큼의 스레드를 생성하여 수행한다.

수행결과

Thread : async-thread-1, read item : 2
Thread : async-thread-2, read item : 1
Thread : async-thread-1, read item : 3
Thread : async-thread-2, read item : 4
Thread : async-thread-1, read item : 5
Thread : async-thread-2, read item : 6
Thread : async-thread-1, read item : 7
Thread : async-thread-2, read item : 8
Thread : async-thread-1, read item : 9
Thread : async-thread-2, read item : 10
Thread : async-thread-1, read item : 11
Thread : async-thread-2, read item : 12
Thread : async-thread-1, read item : 13
Thread : async-thread-2, read item : 14
Thread : async-thread-1, read item : 15
Thread : async-thread-2, read item : 16
Thread : async-thread-1, read item : 17
Thread : async-thread-2, read item : 18
Thread : async-thread-1, read item : 19
Thread : async-thread-2, read item : 20
Thread : async-thread-1, read item : 21
Thread : async-thread-2, read item : 22
Thread : async-thread-1, read item : 23
Thread : async-thread-2, read item : 24
Thread : async-thread-1, read item : 25
Thread : async-thread-2, read item : 26
Thread : async-thread-1, read item : 27
Thread : async-thread-2, read item : 28
Thread : async-thread-1, read item : 29
Thread : async-thread-2, read item : 30
Thread : async-thread-1, read item : 31
Thread : async-thread-2, read item : 32
Thread : async-thread-1, read item : 33
Thread : async-thread-2, read item : 34
Thread : async-thread-1, read item : 35
Thread : async-thread-2, read item : 36
Thread : async-thread-1, read item : 37
Thread : async-thread-2, read item : 38
Thread : async-thread-1, read item : 39
Thread : async-thread-2, read item : 40
Thread : async-thread-1, read item : 41
Thread : async-thread-2, read item : 42
Thread : async-thread-1, read item : 43
Thread : async-thread-2, read item : 44
Thread : async-thread-1, read item : 45
Thread : async-thread-2, read item : 46
Thread : async-thread-1, read item : 47
Thread : async-thread-2, read item : 48
Thread : async-thread-1, read item : 49
Thread : async-thread-2, read item : 50
Thread : async-thread-1, read item : 51
Thread : async-thread-2, read item : 52
Thread : async-thread-1, read item : 53
Thread : async-thread-2, read item : 54
Thread : async-thread-1, read item : 55
Thread : async-thread-2, read item : 56
Thread : async-thread-1, read item : 57
Thread : async-thread-2, read item : 58
Thread : async-thread-1, read item : 59
Thread : async-thread-2, read item : 60
Thread : async-thread-1, read item : 61
Thread : async-thread-2, read item : 62
Thread : async-thread-1, read item : 63
Thread : async-thread-2, read item : 64
Thread : async-thread-1, read item : 65
Thread : async-thread-2, read item : 66
Thread : async-thread-1, read item : 67
Thread : async-thread-2, read item : 68
Thread : async-thread-1, read item : 69
Thread : async-thread-2, read item : 70
Thread : async-thread-1, read item : 71
Thread : async-thread-2, read item : 72
Thread : async-thread-1, read item : 73
Thread : async-thread-2, read item : 74
Thread : async-thread-1, read item : 75
Thread : async-thread-2, read item : 76
Thread : async-thread-1, read item : 77
Thread : async-thread-2, read item : 78
Thread : async-thread-1, read item : 79
Thread : async-thread-2, read item : 80
Thread : async-thread-1, read item : 81
Thread : async-thread-2, read item : 82
Thread : async-thread-1, read item : 83
Thread : async-thread-2, read item : 84
Thread : async-thread-1, read item : 85
Thread : async-thread-2, read item : 1
Thread : async-thread-1, read item : 2
Thread : async-thread-2, read item : 3
Thread : async-thread-1, read item : 4
Thread : async-thread-2, read item : 5
Thread : async-thread-1, read item : 6
Thread : async-thread-2, read item : 7
Thread : async-thread-1, read item : 8
Thread : async-thread-2, read item : 9
Thread : async-thread-1, read item : 10
Thread : async-thread-2, read item : 11
Thread : async-thread-1, read item : 12
Thread : async-thread-2, read item : 13
Thread : async-thread-1, read item : 14
Thread : async-thread-2, read item : 15
Thread : async-thread-1, read item : 16
Thread : async-thread-2, read item : 17
Thread : async-thread-1, read item : 18
Thread : async-thread-2, read item : 19
Thread : async-thread-1, read item : 20
Thread : async-thread-2, read item : 21
Thread : async-thread-1, read item : 22
Thread : async-thread-2, read item : 23
Thread : async-thread-1, read item : 24
Thread : async-thread-2, read item : 25
Thread : async-thread-1, read item : 26
Thread : async-thread-2, read item : 27
Thread : async-thread-1, read item : 28
Thread : async-thread-2, read item : 29
Thread : async-thread-1, read item : 30
Thread : async-thread-2, read item : 31
Thread : async-thread-1, read item : 32
Thread : async-thread-2, read item : 33
Thread : async-thread-1, read item : 34
Thread : async-thread-2, read item : 35
Thread : async-thread-1, read item : 36
Thread : async-thread-2, read item : 37
Thread : async-thread-1, read item : 38
Thread : async-thread-2, read item : 39
Thread : async-thread-1, read item : 40
Thread : async-thread-2, read item : 41
Thread : async-thread-1, read item : 42
Thread : async-thread-2, read item : 43
Thread : async-thread-1, read item : 44
Thread : async-thread-2, read item : 45
Thread : async-thread-1, read item : 46
Thread : async-thread-2, read item : 47
Thread : async-thread-1, read item : 48
Thread : async-thread-2, read item : 49
Thread : async-thread-1, read item : 50
Thread : async-thread-2, read item : 51
Thread : async-thread-1, read item : 52
Thread : async-thread-2, read item : 53
Thread : async-thread-1, read item : 54
Thread : async-thread-2, read item : 55
Thread : async-thread-1, read item : 56
Thread : async-thread-2, read item : 57
Thread : async-thread-1, read item : 58
Thread : async-thread-2, read item : 59
Thread : async-thread-1, read item : 60
Thread : async-thread-2, read item : 61
Thread : async-thread-1, read item : 62
Thread : async-thread-2, read item : 63
Thread : async-thread-1, read item : 64
Thread : async-thread-2, read item : 65
Thread : async-thread-1, read item : 66
Thread : async-thread-2, read item : 67
Thread : async-thread-1, read item : 68
Thread : async-thread-2, read item : 69
Thread : async-thread-1, read item : 70
Thread : async-thread-2, read item : 71
Thread : async-thread-1, read item : 72
Thread : async-thread-2, read item : 73
Thread : async-thread-1, read item : 74
Thread : async-thread-2, read item : 75
Thread : async-thread-1, read item : 76
Thread : async-thread-2, read item : 77
Thread : async-thread-1, read item : 78
Thread : async-thread-2, read item : 79
Thread : async-thread-1, read item : 80
Thread : async-thread-2, read item : 81
Thread : async-thread-1, read item : 82
Thread : async-thread-2, read item : 83
Thread : async-thread-2, read item : 84
Thread : async-thread-1, read item : 85
Thread : async-thread-2, read item : 86
Thread : async-thread-1, read item : 87
Thread : async-thread-2, read item : 88
Thread : async-thread-1, read item : 89
Thread : async-thread-2, read item : 90
Thread : async-thread-1, read item : 91
Thread : async-thread-2, read item : 92
Thread : async-thread-1, read item : 93
Thread : async-thread-2, read item : 94
Thread : async-thread-1, read item : 95
Thread : async-thread-2, read item : 96
Thread : async-thread-1, read item : 97
Thread : async-thread-2, read item : 98
Thread : async-thread-1, read item : 99
Thread : async-thread-2, read item : 100
Thread : async-thread-1, process item : 2
Thread : async-thread-2, process item : 1
Thread : async-thread-1, process item : 3
Thread : async-thread-2, process item : 4
Thread : async-thread-1, process item : 5
Thread : async-thread-2, process item : 6
Thread : async-thread-2, process item : 8
Thread : async-thread-2, process item : 10
Thread : async-thread-1, process item : 7
Thread : async-thread-2, process item : 12
Thread : async-thread-1, process item : 9
Thread : async-thread-2, process item : 14
Thread : async-thread-1, process item : 11
Thread : async-thread-2, process item : 16
Thread : async-thread-1, process item : 13
Thread : async-thread-1, process item : 15
Thread : async-thread-2, process item : 18
Thread : async-thread-1, process item : 17
Thread : async-thread-2, process item : 20
Thread : async-thread-1, process item : 19
Thread : async-thread-2, process item : 22
Thread : async-thread-1, process item : 21
Thread : async-thread-2, process item : 24
Thread : async-thread-1, process item : 23
Thread : async-thread-2, process item : 26
Thread : async-thread-1, process item : 25
Thread : async-thread-2, process item : 28
Thread : async-thread-1, process item : 27
Thread : async-thread-2, process item : 30
Thread : async-thread-1, process item : 29
Thread : async-thread-2, process item : 32
Thread : async-thread-1, process item : 31
Thread : async-thread-2, process item : 34
Thread : async-thread-1, process item : 33
Thread : async-thread-2, process item : 36
Thread : async-thread-1, process item : 35
Thread : async-thread-2, process item : 38
Thread : async-thread-1, process item : 37
Thread : async-thread-2, process item : 40
Thread : async-thread-1, process item : 39
Thread : async-thread-2, process item : 42
Thread : async-thread-1, process item : 41
Thread : async-thread-2, process item : 44
Thread : async-thread-1, process item : 43
Thread : async-thread-2, process item : 46
Thread : async-thread-1, process item : 45
Thread : async-thread-2, process item : 48
Thread : async-thread-1, process item : 47
Thread : async-thread-2, process item : 50
Thread : async-thread-1, process item : 49
Thread : async-thread-2, process item : 52
Thread : async-thread-1, process item : 51
Thread : async-thread-1, process item : 53
Thread : async-thread-1, process item : 55
Thread : async-thread-2, process item : 54
Thread : async-thread-1, process item : 57
Thread : async-thread-2, process item : 56
Thread : async-thread-1, process item : 59
Thread : async-thread-2, process item : 58
Thread : async-thread-1, process item : 61
Thread : async-thread-2, process item : 60
Thread : async-thread-1, process item : 63
Thread : async-thread-2, process item : 62
Thread : async-thread-1, process item : 65
Thread : async-thread-2, process item : 64
Thread : async-thread-1, process item : 67
Thread : async-thread-1, process item : 69
Thread : async-thread-2, process item : 66
Thread : async-thread-1, process item : 71
Thread : async-thread-2, process item : 68
Thread : async-thread-1, process item : 73
Thread : async-thread-2, process item : 70
Thread : async-thread-1, process item : 75
Thread : async-thread-2, process item : 72
Thread : async-thread-1, process item : 77
Thread : async-thread-2, process item : 74
Thread : async-thread-1, process item : 79
Thread : async-thread-2, process item : 76
Thread : async-thread-1, process item : 81
Thread : async-thread-2, process item : 78
Thread : async-thread-1, process item : 83
Thread : async-thread-2, process item : 80
Thread : async-thread-1, process item : 85
Thread : async-thread-2, process item : 82
Thread : async-thread-1, process item : 2
Thread : async-thread-2, process item : 84
Thread : async-thread-1, process item : 4
Thread : async-thread-2, process item : 1
Thread : async-thread-1, process item : 6
Thread : async-thread-2, process item : 3
Thread : async-thread-1, process item : 8
Thread : async-thread-2, process item : 5
Thread : async-thread-1, process item : 10
Thread : async-thread-2, process item : 7
Thread : async-thread-1, process item : 12
Thread : async-thread-2, process item : 9
Thread : async-thread-1, process item : 14
Thread : async-thread-2, process item : 11
Thread : async-thread-1, process item : 16
Thread : async-thread-2, process item : 13
Thread : async-thread-1, process item : 18
Thread : async-thread-2, process item : 15
Thread : async-thread-1, process item : 20
Thread : async-thread-2, process item : 17
Thread : async-thread-1, process item : 22
Thread : async-thread-2, process item : 19
Thread : async-thread-1, process item : 24
Thread : async-thread-2, process item : 21
Thread : async-thread-1, process item : 26
Thread : async-thread-2, process item : 23
Thread : async-thread-1, process item : 28
Thread : async-thread-2, process item : 25
Thread : async-thread-2, process item : 27
Thread : async-thread-2, process item : 29
Thread : async-thread-1, process item : 30
Thread : async-thread-2, process item : 31
Thread : async-thread-1, process item : 32
Thread : async-thread-2, process item : 33
Thread : async-thread-1, process item : 34
Thread : async-thread-2, process item : 35
Thread : async-thread-1, process item : 36
Thread : async-thread-2, process item : 37
Thread : async-thread-1, process item : 38
Thread : async-thread-2, process item : 39
Thread : async-thread-2, process item : 41
Thread : async-thread-2, process item : 43
Thread : async-thread-1, process item : 40
Thread : async-thread-2, process item : 45
Thread : async-thread-1, process item : 42
Thread : async-thread-2, process item : 47
Thread : async-thread-1, process item : 44
Thread : async-thread-2, process item : 49
Thread : async-thread-1, process item : 46
Thread : async-thread-1, process item : 48
Thread : async-thread-1, process item : 50
Thread : async-thread-2, process item : 51
Thread : async-thread-1, process item : 52
Thread : async-thread-2, process item : 53
Thread : async-thread-1, process item : 54
Thread : async-thread-2, process item : 55
Thread : async-thread-1, process item : 56
Thread : async-thread-2, process item : 57
Thread : async-thread-1, process item : 58
Thread : async-thread-2, process item : 59
Thread : async-thread-1, process item : 60
Thread : async-thread-2, process item : 61
Thread : async-thread-1, process item : 62
Thread : async-thread-2, process item : 63
Thread : async-thread-1, process item : 64
Thread : async-thread-2, process item : 65
Thread : async-thread-1, process item : 66
Thread : async-thread-2, process item : 67
Thread : async-thread-1, process item : 68
Thread : async-thread-2, process item : 69
Thread : async-thread-1, process item : 70
Thread : async-thread-2, process item : 71
Thread : async-thread-1, process item : 72
Thread : async-thread-2, process item : 73
Thread : async-thread-1, process item : 74
Thread : async-thread-2, process item : 75
Thread : async-thread-1, process item : 76
Thread : async-thread-2, process item : 77
Thread : async-thread-1, process item : 78
Thread : async-thread-2, process item : 79
Thread : async-thread-1, process item : 80
Thread : async-thread-2, process item : 81
Thread : async-thread-1, process item : 82
Thread : async-thread-2, process item : 83
Thread : async-thread-2, process item : 84
Thread : async-thread-1, process item : 85
Thread : async-thread-2, process item : 86
Thread : async-thread-1, process item : 87
Thread : async-thread-2, process item : 88
Thread : async-thread-1, process item : 89
Thread : async-thread-2, process item : 90
Thread : async-thread-1, process item : 91
Thread : async-thread-2, process item : 92
Thread : async-thread-1, process item : 93
Thread : async-thread-2, process item : 94
Thread : async-thread-1, process item : 95
Thread : async-thread-2, process item : 96
Thread : async-thread-1, process item : 97
Thread : async-thread-2, process item : 98
Thread : async-thread-1, process item : 99
Thread : async-thread-2, process item : 100

ItemReader가 동기화되어야하는 이유

JdbcCursorItemReader

package com.project.springbatch._43_multiThread;

import lombok.RequiredArgsConstructor;
import org.springframework.batch.core.Job;
import org.springframework.batch.core.Step;
import org.springframework.batch.core.configuration.annotation.JobBuilderFactory;
import org.springframework.batch.core.configuration.annotation.StepBuilderFactory;
import org.springframework.batch.core.launch.support.RunIdIncrementer;
import org.springframework.batch.item.ItemProcessor;
import org.springframework.batch.item.database.*;
import org.springframework.batch.item.database.builder.JdbcCursorItemReaderBuilder;
import org.springframework.batch.item.database.support.MySqlPagingQueryProvider;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.core.task.TaskExecutor;
import org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor;

import javax.sql.DataSource;
import java.util.HashMap;
import java.util.Map;

/*
--job.name=multiThreadJob
*/
@RequiredArgsConstructor
@Configuration
public class MultiThreadStepConfiguration {

    private final JobBuilderFactory jobBuilderFactory;
    private final StepBuilderFactory stepBuilderFactory;
    private final DataSource dataSource;

    @Bean
    public Job multiThreadJob() throws Exception {
        return jobBuilderFactory.get("multiThreadJob")
                .incrementer(new RunIdIncrementer())
                .start(multiThreadStep1())
                .listener(new MultiThreadStopWatchJobListener())
                .build();
    }

    @Bean
    public Step multiThreadStep1() throws Exception {
        return stepBuilderFactory.get("multiThreadStep1")
                .<MultiThreadCustomer, MultiThreadCustomer>chunk(100)
                .reader(multiThreadCustomCursorItemReader())
                .listener(new MultiThreadCustomReadListener())
                .processor((ItemProcessor<MultiThreadCustomer, MultiThreadCustomer>) item -> item)
                .listener(new MultiThreadCustomProcessListener())
                .writer(multiThreadCustomItemWriter())
                .listener(new MultiThreadCustomWriteListener())
                .taskExecutor(multiThreadTaskExecutor())
                .throttleLimit(2)
                .build();
    }

    @Bean
    public JdbcCursorItemReader<MultiThreadCustomer> multiThreadCustomCursorItemReader() {
        return new JdbcCursorItemReaderBuilder()
                .name("jdbcCursorItemReader")
                .fetchSize(100)
                .sql("select id, firstName, lastName, birthdate from customer order by id")
                .beanRowMapper(MultiThreadCustomer.class)
                .dataSource(dataSource)
                .build();
    }

    @Bean
    public JdbcBatchItemWriter multiThreadCustomItemWriter() {
        JdbcBatchItemWriter<MultiThreadCustomer> itemWriter = new JdbcBatchItemWriter<>();

        itemWriter.setDataSource(this.dataSource);
        itemWriter.setSql("insert into customer2 values (:id, :firstName, :lastName, :birthdate)");
        itemWriter.setItemSqlParameterSourceProvider(new BeanPropertyItemSqlParameterSourceProvider());
        itemWriter.afterPropertiesSet();

        return itemWriter;
    }

    @Bean
    public TaskExecutor multiThreadTaskExecutor(){
        ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
        executor.setCorePoolSize(4); // 몇개의 스레드를 관리할것인지?
        executor.setMaxPoolSize(8); // 4개의 스레드가 작업을 처리하고 있는데, 나머지 처리되지 않은 task가 있을때 얼마만큼의 최대 스레드를 생성할것인지?
        executor.setThreadNamePrefix("async-thread-");
        return executor;
    }
}

JdbcCursorItemReader는 동기화가 안되기 때문에 중복 데이터가 발생한다. 따라서 아래와 같이 실행결과에서 오류가 함께 발생한다.

따라서 위의 예제처럼 JdbcPagingItemReader을 사용해야한다.

관련하여 아래의 포스팅을 참고하자.

JdbcCursorItemReader

https://devfunny.tistory.com/827

[SringBatch 실습] 26. Cursor - JdbcCursorItemReader, JpaCursorItemReader

Cursor 방식 JDBC ResultSet의 기본 메커니즘을 사용한다. 현재 행에 커서를 유지하며 다음 데이터를 호출하면 다음 행으로 커서를 이동하며 데이터 반환이 이루어진다. (Streaming 방식) ResultSet이 open()

devfunny.tistory.com

JdbcPagingItemReader

https://devfunny.tistory.com/828

[SringBatch 실습] 27. Paging - JdbcPagingItemReader, JpaPagingItemReader

Paging 방식 페이징 단위로 데이터를 조회하는 방식으로 Page Size만큼 한번에 메모리로 가지고 온 다음 한개씩 읽는다. 한 페이지를 읽을때마다 Connection을 맺고 끊기 때문에 대량의 데이터를 처리

devfunny.tistory.com

수행결과

Thread : async-thread-3, read item : 3
Thread : async-thread-1, read item : 2
Thread : async-thread-3, read item : 4
Thread : async-thread-1, read item : 4
Thread : async-thread-1, read item : 6
Thread : async-thread-1, read item : 7
Thread : async-thread-1, read item : 8
Thread : async-thread-1, read item : 9
Thread : async-thread-1, read item : 10
Thread : async-thread-1, read item : 11
Thread : async-thread-1, read item : 12
Thread : async-thread-1, read item : 13
Thread : async-thread-1, read item : 14
Thread : async-thread-1, read item : 15
Thread : async-thread-1, read item : 16
Thread : async-thread-1, read item : 17
Thread : async-thread-1, read item : 18
Thread : async-thread-1, read item : 19
Thread : async-thread-1, read item : 20
Thread : async-thread-1, read item : 21
Thread : async-thread-1, read item : 22
Thread : async-thread-1, read item : 23
Thread : async-thread-1, read item : 24
Thread : async-thread-1, read item : 25
Thread : async-thread-1, read item : 26
Thread : async-thread-1, read item : 27
Thread : async-thread-1, read item : 28
Thread : async-thread-1, read item : 29
Thread : async-thread-1, read item : 30
Thread : async-thread-1, read item : 31
Thread : async-thread-1, read item : 32
Thread : async-thread-1, read item : 33
Thread : async-thread-1, read item : 34
Thread : async-thread-1, read item : 35
Thread : async-thread-1, read item : 36
Thread : async-thread-1, read item : 37
Thread : async-thread-1, read item : 38
Thread : async-thread-1, read item : 39
Thread : async-thread-1, read item : 40
Thread : async-thread-1, read item : 41
Thread : async-thread-1, read item : 42
Thread : async-thread-1, read item : 43
Thread : async-thread-1, read item : 44
Thread : async-thread-1, read item : 45
Thread : async-thread-1, read item : 46
Thread : async-thread-1, read item : 47
Thread : async-thread-1, read item : 48
Thread : async-thread-1, read item : 49
Thread : async-thread-1, read item : 50
Thread : async-thread-1, read item : 51
Thread : async-thread-1, read item : 52
Thread : async-thread-1, read item : 53
Thread : async-thread-1, read item : 54
Thread : async-thread-1, read item : 55
Thread : async-thread-1, read item : 56
Thread : async-thread-1, read item : 57
Thread : async-thread-1, read item : 58
Thread : async-thread-1, read item : 59
Thread : async-thread-1, read item : 60
Thread : async-thread-1, read item : 61
Thread : async-thread-1, read item : 62
Thread : async-thread-1, read item : 63
Thread : async-thread-1, read item : 64
Thread : async-thread-1, read item : 65
Thread : async-thread-1, read item : 66
Thread : async-thread-1, read item : 67
Thread : async-thread-1, read item : 68
Thread : async-thread-1, read item : 69
Thread : async-thread-1, read item : 70
Thread : async-thread-1, read item : 71
Thread : async-thread-1, read item : 72
Thread : async-thread-1, read item : 73
Thread : async-thread-1, read item : 74
Thread : async-thread-1, read item : 75
Thread : async-thread-1, read item : 76
Thread : async-thread-1, read item : 77
Thread : async-thread-1, read item : 78
Thread : async-thread-1, read item : 79
Thread : async-thread-1, read item : 80
Thread : async-thread-1, read item : 81
Thread : async-thread-1, read item : 82
Thread : async-thread-1, read item : 83
Thread : async-thread-1, read item : 84
Thread : async-thread-1, read item : 85
Thread : async-thread-1, read item : 86
Thread : async-thread-1, read item : 87
Thread : async-thread-1, read item : 88
Thread : async-thread-1, read item : 89
Thread : async-thread-1, read item : 90
Thread : async-thread-1, read item : 91
Thread : async-thread-1, read item : 92
Thread : async-thread-1, read item : 93
Thread : async-thread-1, read item : 94
Thread : async-thread-1, read item : 95
Thread : async-thread-1, read item : 96
Thread : async-thread-1, read item : 97
Thread : async-thread-1, read item : 98
Thread : async-thread-1, read item : 99
Thread : async-thread-1, read item : 100
Thread : async-thread-1, process item : 2
Thread : async-thread-1, process item : 4
Thread : async-thread-1, process item : 6
Thread : async-thread-1, process item : 7
Thread : async-thread-1, process item : 8
Thread : async-thread-1, process item : 9
Thread : async-thread-1, process item : 10
Thread : async-thread-1, process item : 11
Thread : async-thread-1, process item : 12
Thread : async-thread-1, process item : 13
Thread : async-thread-1, process item : 14
Thread : async-thread-1, process item : 15
Thread : async-thread-1, process item : 16
Thread : async-thread-1, process item : 17
Thread : async-thread-1, process item : 18
Thread : async-thread-1, process item : 19
Thread : async-thread-1, process item : 20
Thread : async-thread-1, process item : 21
Thread : async-thread-1, process item : 22
Thread : async-thread-1, process item : 23
Thread : async-thread-1, process item : 24
Thread : async-thread-1, process item : 25
Thread : async-thread-1, process item : 26
Thread : async-thread-1, process item : 27
Thread : async-thread-1, process item : 28
Thread : async-thread-1, process item : 29
Thread : async-thread-1, process item : 30
Thread : async-thread-1, process item : 31
Thread : async-thread-1, process item : 32
Thread : async-thread-1, process item : 33
Thread : async-thread-1, process item : 34
Thread : async-thread-1, process item : 35
Thread : async-thread-1, process item : 36
Thread : async-thread-1, process item : 37
Thread : async-thread-1, process item : 38
Thread : async-thread-1, process item : 39
Thread : async-thread-1, process item : 40
Thread : async-thread-1, process item : 41
Thread : async-thread-1, process item : 42
Thread : async-thread-1, process item : 43
Thread : async-thread-1, process item : 44
Thread : async-thread-1, process item : 45
Thread : async-thread-1, process item : 46
Thread : async-thread-1, process item : 47
Thread : async-thread-1, process item : 48
Thread : async-thread-1, process item : 49
Thread : async-thread-1, process item : 50
Thread : async-thread-1, process item : 51
Thread : async-thread-1, process item : 52
Thread : async-thread-1, process item : 53
Thread : async-thread-1, process item : 54
Thread : async-thread-1, process item : 55
Thread : async-thread-1, process item : 56
Thread : async-thread-1, process item : 57
Thread : async-thread-1, process item : 58
Thread : async-thread-1, process item : 59
Thread : async-thread-1, process item : 60
Thread : async-thread-1, process item : 61
Thread : async-thread-1, process item : 62
Thread : async-thread-1, process item : 63
Thread : async-thread-1, process item : 64
Thread : async-thread-1, process item : 65
Thread : async-thread-1, process item : 66
Thread : async-thread-1, process item : 67
Thread : async-thread-1, process item : 68
Thread : async-thread-1, process item : 69
Thread : async-thread-1, process item : 70
Thread : async-thread-1, process item : 71
Thread : async-thread-1, process item : 72
Thread : async-thread-1, process item : 73
Thread : async-thread-1, process item : 74
Thread : async-thread-1, process item : 75
Thread : async-thread-1, process item : 76
Thread : async-thread-1, process item : 77
Thread : async-thread-1, process item : 78
Thread : async-thread-1, process item : 79
Thread : async-thread-1, process item : 80
Thread : async-thread-1, process item : 81
Thread : async-thread-1, process item : 82
Thread : async-thread-1, process item : 83
Thread : async-thread-1, process item : 84
Thread : async-thread-1, process item : 85
Thread : async-thread-1, process item : 86
Thread : async-thread-1, process item : 87
Thread : async-thread-1, process item : 88
Thread : async-thread-1, process item : 89
Thread : async-thread-1, process item : 90
Thread : async-thread-1, process item : 91
Thread : async-thread-1, process item : 92
Thread : async-thread-1, process item : 93
Thread : async-thread-1, process item : 94
Thread : async-thread-1, process item : 95
Thread : async-thread-1, process item : 96
Thread : async-thread-1, process item : 97
Thread : async-thread-1, process item : 98
Thread : async-thread-1, process item : 99
Thread : async-thread-1, process item : 100
Thread : async-thread-1, write items : 97
org.springframework.dao.InvalidDataAccessResourceUsageException: Unexpected cursor position change.

저작자표시

'Coding > Spring Batch' 카테고리의 다른 글

[SpringBatch 실습] 32. 스프링 배치 Partitioning(파티셔닝) Job 수행하여 분석 (0)	2022.07.12
[SpringBatch 실습] 31. 스프링 배치 Parallel Steps(병렬) 수행하여 분석 (0)	2022.07.11
[SpringBatch 실습] 29. 스프링 배치의 단일 스레드 vs 멀티 스레드 Job 수행하여 분석 (0)	2022.07.06
[SpringBatch 실습] 28. ItemReaderAdapter (0)	2022.07.05
[SpringBatch 실습] 27. Paging - JdbcPagingItemReader, JpaPagingItemReader (0)	2022.07.04

[SpringBatch 실습] 30. 스프링 배치 MultiThread(멀티 스레드) Job 수행하여 분석

스프링배치 Multi-thread

예제코드

Listener 구성

Job 생성

수행결과

디버깅

throttleLimit() 설정해보자.

ItemReader가 동기화되어야하는 이유

'Coding > Spring Batch' 카테고리의 다른 글

Copyright © bada All Rights Reserved

Designed by JB FACTORY

« 2025/01 »
일	월	화	수	목	금	토
			1	2	3	4
5	6	7	8	9	10	11
12	13	14	15	16	17	18
19	20	21	22	23	24	25
26	27	28	29	30	31

스프링배치 Multi-thread

예제코드

Listener 구성

Job 생성

수행결과

디버깅

throttleLimit() 설정해보자.

ItemReader가 동기화되어야하는 이유

'Coding > Spring Batch' 카테고리의 다른 글

'Coding/Spring Batch' 관련 글

[SpringBatch 실습] 32. 스프링 배치 Partitioning(파티셔닝) Job 수행하여 분석

[SpringBatch 실습] 31. 스프링 배치 Parallel Steps(병렬) 수행하여 분석

[SpringBatch 실습] 29. 스프링 배치의 단일 스레드 vs 멀티 스레드 Job 수행하여 분석

[SpringBatch 실습] 28. ItemReaderAdapter

Copyright © bada All Rights Reserved

Designed by JB FACTORY

티스토리툴바