如何使用spring JDBC Template批量在數據庫中執行多個插入？

Question

我需要在數據庫中一次插入數千條記錄。我在我的應用程序中使用Spring JDBC模板。

以下是我迄今爲止編寫的代碼，它一次執行所有插入操作。所以，如果我有10,000個用戶，他們就一次插入。但我想要的是批量執行它們，例如一批中的500條記錄等等。

@Override 
public void saveBatch(final List<Employee> employeeList) { 
    final int batchSize = 500; 

    getJdbcTemplate().batchUpdate(QUERY_SAVE, 
      new BatchPreparedStatementSetter() { 
       @Override 
       public void setValues(PreparedStatement ps, int i) 
         throws SQLException { 
        Employee employee = employeeList.get(i); 
        ps.setString(1, employee.getFirstname()); 
        ps.setString(2, employee.getLastname()); 
        ps.setString(3, employee.getEmployeeIdOnSourceSystem()); 
       } 

       @Override 
       public int getBatchSize() { 
        return employeeList.size(); 
       } 
      }); 

} 

如何更改上面的代碼，以便代替employeeList.size（）作爲批量大小，我們可以有批量大小爲500說，執行它們，然後接下來的500等等？

請幫忙。

Answer 1

我不確定您是否可以單獨使用JDBC模板來執行此操作。也許你可以逐步調用batchUpdate方法，將大列表分割爲批量大小的塊。

看一看這裏：

@Override 
public void saveBatch(final List<Employee> employeeList) { 
    final int batchSize = 500; 

    for (int j = 0; j < employeeList.size(); j += batchSize) { 

     final List<Employee> batchList = employeeList.subList(j, j + batchSize > employeeList.size() ? employeeList.size() : j + batchSize); 

     getJdbcTemplate().batchUpdate(QUERY_SAVE, 
      new BatchPreparedStatementSetter() { 
       @Override 
       public void setValues(PreparedStatement ps, int i) 
         throws SQLException { 
        Employee employee = batchList.get(i); 
        ps.setString(1, employee.getFirstname()); 
        ps.setString(2, employee.getLastname()); 
        ps.setString(3, employee.getEmployeeIdOnSourceSystem()); 
       } 

       @Override 
       public int getBatchSize() { 
        return batchList.size(); 
       } 
      }); 

    } 
} 

Answer 2

我知道這是一個有點晚了，但你可以做類似於@adarshr是做什麼的，除非使用Google Guava Lists.partition得到子列表。

public void saveBatch(final List<Employee> employeeList) { 
    final int batchSize = 500; 
    List<List<Employee>> batchLists = Lists.partition(employeeList, batchSize); 

    for(List<Employee> batch : batchLists) { 
     getJdbcTemplate().batchUpdate(QUERY_SAVE, new BatchPreparedStatementSetter() { 
      @Override 
      public void setValues(PreparedStatement ps, int i) 
        throws SQLException { 
       Employee employee = batch.get(i); 
       ps.setString(1, employee.getFirstname()); 
       ps.setString(2, employee.getLastname()); 
       ps.setString(3, employee.getEmployeeIdOnSourceSystem()); 
      } 

      @Override 
      public int getBatchSize() { 
       return batch.size(); 
      } 
     }); 
    } 
} 

Answer 3

還是簡單的方式是修改getBatchsize（）方法如下效果很好

無需分區或列表:)的子集，

@Override 
public void saveBatch(final List<Employee> employeeList) { 
    final int batchSize = 500; 
    getJdbcTemplate().batchUpdate(QUERY_SAVE, 
      new BatchPreparedStatementSetter() { 
       @Override 
       public void setValues(PreparedStatement ps, int i) 
         throws SQLException { 
        Employee employee = employeeList.get(i); 
        ps.setString(1, employee.getFirstname()); 
        ps.setString(2, employee.getLastname()); 
        ps.setString(3, employee.getEmployeeIdOnSourceSystem()); 
       } 

       @Override 
       public int getBatchSize() { 
        if (batchSize > employeeList.size()) { 
         return employeeList.size(); 
        } 
        return batchSize; 
       } 
      }); 
} 

Answer 4

Spring provide Batch operations with multiple batches 
here 100 is batch size. 


public class JdbcActorDao implements ActorDao { 

    private JdbcTemplate jdbcTemplate; 

    public void setDataSource(DataSource dataSource) { 
     this.jdbcTemplate = new JdbcTemplate(dataSource); 
    } 

    public int[][] batchUpdate(final Collection<Actor> actors) { 
     int[][] updateCounts = jdbcTemplate.batchUpdate(
       "update t_actor set first_name = ?, last_name = ? where id = ?", 
       actors, 
       100, 
       new ParameterizedPreparedStatementSetter<Actor>() { 
        public void setValues(PreparedStatement ps, Actor argument) throws SQLException { 
         ps.setString(1, argument.getFirstName()); 
         ps.setString(2, argument.getLastName()); 
         ps.setLong(3, argument.getId().longValue()); 
        } 
       }); 
     return updateCounts; 
    } 

    // ... additional methods 

}