18
我想將大量實體插入到交叉表中。爲此，我想啓用hibernate的批量插入選項，以便每次插入一次不會添加1行，但會添加20行或50行。我想重現hibernate's tutorial的結果。我創建了一個測試函數，試圖插入30個客戶，就像在示例中一樣：如何使用Hibernate和Spring Boot啓用批量插入
//Batch inserting test entities
//This code is inside transaction already, so I'm not creating a new one
entityManager.flush();
for (int i = 0; i < 30; ++i) {
Customer customer = new Customer("Customer" + i, (i + 5) * 1000, "Position " + i);
entityManager.persist(customer);
if (i % 20 == 0) {
entityManager.flush();
entityManager.clear();
}
}
entityManager.flush();
entityManager.clear();
Customer類沒有生成的ID。
@Entity
public class Customer {
@Id
private String id;
@Column
private String name;
@Column
private long salary;
@Column
private String position;
public Customer() {
}
public Customer(String name, long salary, String position) {
this.id = UUID.randomUUID().toString();
this.name = name;
this.salary = salary;
this.position = position;
}
}
我期待看到的是2個插入語句，一個用於20個記錄，另一個用於10個記錄。但是，當我打開postgres日誌時，我看到30條插入語句，每條語句只插入1行。我仔細檢查過可以使用postgres插入多行。
我認爲這個問題是由hibernate.jdbc.batch_size 20參數引起的，我應該把它傳遞給hibernate。但是，由於我使用的是spring引導，我唯一的配置文件是application.properties。所以，我已經試過有將其插入：
hibernate.jdbc.batch_size=20
hibernate.order_inserts=true
hibernate.order_updates=true
hibernate.jdbc.batch_versioned_data=true
spring.jpa.hibernate.jdbc.batch_size=20
spring.jpa.hibernate.order_inserts=true
spring.jpa.hibernate.order_updates=true
spring.jpa.hibernate.jdbc.batch_versioned_data=true
spring.jpa.properties.hibernate.jdbc.batch_size=20
spring.jpa.properties.hibernate.order_inserts=true
spring.jpa.properties.hibernate.order_updates=true
spring.jpa.properties.hibernate.jdbc.batch_versioned_data=true
根據這一answer，這應該是足夠了，但 我也試着複製這些參數到hibernate.properties文件我的罐子裏，並通過命令行提供的參數：-Dhibernate.jdbc.batch_size=20如在此documentation中所述。
這一切都沒有幫助。
我也找不到在我的代碼中讀取批量大小屬性的方法。看起來，在spring引導應用程序中不存在hibernate文檔中提到的大多數對象。
如何在spring引導應用程序中啓用hibernate的批量插入？
我創建了一個最小的工作示例抄錄問題： https://github.com/Alexey-/spring-boot-batch
從弗拉德·米哈爾西亞接收答案，我意識到，是的，事實上批量報表工作後，至少在休眠和jdbc級別。
但是，postgres日誌本身顯示出非常有趣的行爲：它們對於批量插入和普通插入的情況幾乎相同。
我最初希望看到的是，休眠將使用的語句是這樣的：
test=# INSERT INTO customer (name, position, salary, id) values ('CustomerX', 'PositionX', '1000', 'idX'), ('CUSTOMERY', 'POSITIONY', '1000', 'idY');
這將產生日誌類似記錄本：
2015-12-15 11:43:33.238 MSK LOG: statement: INSERT INTO customer (name, position, salary, id) values ('CustomerX', 'PositionX', '1000', 'idX'), ('CUSTOMERY', 'POSITIONY', '1000', 'idY');
然而，這並非如此。
當批量插入啓用（P6SPY顯示語句，其實批處理），Postgres將產生類似這樣的記錄：
2015-12-15 12:07:00.638 MSK LOG: execute <unnamed>: BEGIN
2015-12-15 12:07:00.638 MSK LOG: duration: 0.000 ms
2015-12-15 12:07:00.638 MSK LOG: duration: 0.000 ms parse <unnamed>: insert into customer (name, position, salary, id) values ($1, $2, $3, $4)
2015-12-15 12:07:00.638 MSK LOG: duration: 0.000 ms bind <unnamed>: insert into customer (name, position, salary, id) values ($1, $2, $3, $4)
2015-12-15 12:07:00.638 MSK DETAIL: parameters: $1 = 'Customer0', $2 = 'Position 0', $3 = '0', $4 = '9c6a86fb-c991-4e98-aa65-fa736ef67dd7'
2015-12-15 12:07:00.638 MSK LOG: execute <unnamed>: insert into customer (name, position, salary, id) values ($1, $2, $3, $4)
2015-12-15 12:07:00.638 MSK DETAIL: parameters: $1 = 'Customer0', $2 = 'Position 0', $3 = '0', $4 = '9c6a86fb-c991-4e98-aa65-fa736ef67dd7'
2015-12-15 12:07:00.639 MSK LOG: duration: 1.000 ms
2015-12-15 12:07:00.648 MSK LOG: duration: 0.000 ms parse S_1: insert into customer (name, position, salary, id) values ($1, $2, $3, $4)
2015-12-15 12:07:00.648 MSK LOG: duration: 0.000 ms bind S_1: insert into customer (name, position, salary, id) values ($1, $2, $3, $4)
2015-12-15 12:07:00.648 MSK DETAIL: parameters: $1 = 'Customer1', $2 = 'Position 1', $3 = '10', $4 = 'c8b2669c-044a-4a4d-acbd-31c3bcd9a783'
2015-12-15 12:07:00.648 MSK LOG: execute S_1: insert into customer (name, position, salary, id) values ($1, $2, $3, $4)
2015-12-15 12:07:00.648 MSK DETAIL: parameters: $1 = 'Customer1', $2 = 'Position 1', $3 = '10', $4 = 'c8b2669c-044a-4a4d-acbd-31c3bcd9a783'
2015-12-15 12:07:00.648 MSK LOG: duration: 0.000 ms
2015-12-15 12:07:00.648 MSK LOG: duration: 0.000 ms bind S_1: insert into customer (name, position, salary, id) values ($1, $2, $3, $4)
2015-12-15 12:07:00.648 MSK DETAIL: parameters: $1 = 'Customer2', $2 = 'Position 2', $3 = '20', $4 = '1c694f41-2ce7-4ee2-a0c0-f359690506f0'
2015-12-15 12:07:00.649 MSK LOG: execute S_1: insert into customer (name, position, salary, id) values ($1, $2, $3, $4)
2015-12-15 12:07:00.649 MSK DETAIL: parameters: $1 = 'Customer2', $2 = 'Position 2', $3 = '20', $4 = '1c694f41-2ce7-4ee2-a0c0-f359690506f0'
2015-12-15 12:07:00.649 MSK LOG: duration: 0.000 ms
2015-12-15 12:07:00.649 MSK LOG: duration: 0.000 ms bind S_1: insert into customer (name, position, salary, id) values ($1, $2, $3, $4)
2015-12-15 12:07:00.649 MSK DETAIL: parameters: $1 = 'Customer3', $2 = 'Position 3', $3 = '30', $4 = '1815947d-2604-48d4-a6be-43f6905130cf'
2015-12-15 12:07:00.649 MSK LOG: execute S_1: insert into customer (name, position, salary, id) values ($1, $2, $3, $4)
2015-12-15 12:07:00.649 MSK DETAIL: parameters: $1 = 'Customer3', $2 = 'Position 3', $3 = '30', $4 = '1815947d-2604-48d4-a6be-43f6905130cf'
2015-12-15 12:07:00.649 MSK LOG: duration: 0.000 ms
2015-12-15 12:07:00.649 MSK LOG: duration: 0.000 ms bind S_1: insert into customer (name, position, salary, id) values ($1, $2, $3, $4)
2015-12-15 12:07:00.649 MSK DETAIL: parameters: $1 = 'Customer4', $2 = 'Position 4', $3 = '40', $4 = 'cc521007-820f-4d58-8e1a-16a166aa91cf'
2015-12-15 12:07:00.649 MSK LOG: execute S_1: insert into customer (name, position, salary, id) values ($1, $2, $3, $4)
2015-12-15 12:07:00.649 MSK DETAIL: parameters: $1 = 'Customer4', $2 = 'Position 4', $3 = '40', $4 = 'cc521007-820f-4d58-8e1a-16a166aa91cf'
2015-12-15 12:07:00.649 MSK LOG: duration: 0.000 ms
... the rest of the logs is identical and do not provide any valuable information...
當批處理語句被禁用（P6SPY表明，沒有配料執行） ，日誌將是這樣的：
2015-12-15 12:09:00.246 MSK LOG: execute <unnamed>: BEGIN
2015-12-15 12:09:00.246 MSK LOG: duration: 0.000 ms
2015-12-15 12:09:00.246 MSK LOG: duration: 0.000 ms parse <unnamed>: insert into customer (name, position, salary, id) values ($1, $2, $3, $4)
2015-12-15 12:09:00.246 MSK LOG: duration: 0.000 ms bind <unnamed>: insert into customer (name, position, salary, id) values ($1, $2, $3, $4)
2015-12-15 12:09:00.246 MSK DETAIL: parameters: $1 = 'Customer0', $2 = 'Position 0', $3 = '0', $4 = '9e085ad0-437f-4d7d-afaa-e342e031cbee'
2015-12-15 12:09:00.246 MSK LOG: execute <unnamed>: insert into customer (name, position, salary, id) values ($1, $2, $3, $4)
2015-12-15 12:09:00.246 MSK DETAIL: parameters: $1 = 'Customer0', $2 = 'Position 0', $3 = '0', $4 = '9e085ad0-437f-4d7d-afaa-e342e031cbee'
2015-12-15 12:09:00.246 MSK LOG: duration: 0.000 ms
2015-12-15 12:09:00.248 MSK LOG: duration: 0.000 ms parse <unnamed>: insert into customer (name, position, salary, id) values ($1, $2, $3, $4)
2015-12-15 12:09:00.248 MSK LOG: duration: 0.000 ms bind <unnamed>: insert into customer (name, position, salary, id) values ($1, $2, $3, $4)
2015-12-15 12:09:00.248 MSK DETAIL: parameters: $1 = 'Customer1', $2 = 'Position 1', $3 = '10', $4 = 'f29cfa40-7d24-49a6-ae5d-2a2021932d80'
2015-12-15 12:09:00.248 MSK LOG: execute <unnamed>: insert into customer (name, position, salary, id) values ($1, $2, $3, $4)
2015-12-15 12:09:00.248 MSK DETAIL: parameters: $1 = 'Customer1', $2 = 'Position 1', $3 = '10', $4 = 'f29cfa40-7d24-49a6-ae5d-2a2021932d80'
2015-12-15 12:09:00.249 MSK LOG: duration: 1.000 ms
2015-12-15 12:09:00.250 MSK LOG: duration: 0.000 ms parse <unnamed>: insert into customer (name, position, salary, id) values ($1, $2, $3, $4)
2015-12-15 12:09:00.250 MSK LOG: duration: 0.000 ms bind <unnamed>: insert into customer (name, position, salary, id) values ($1, $2, $3, $4)
2015-12-15 12:09:00.250 MSK DETAIL: parameters: $1 = 'Customer2', $2 = 'Position 2', $3 = '20', $4 = '067dd6d4-5060-467f-b533-75994ecbaedc'
2015-12-15 12:09:00.250 MSK LOG: execute <unnamed>: insert into customer (name, position, salary, id) values ($1, $2, $3, $4)
2015-12-15 12:09:00.250 MSK DETAIL: parameters: $1 = 'Customer2', $2 = 'Position 2', $3 = '20', $4 = '067dd6d4-5060-467f-b533-75994ecbaedc'
2015-12-15 12:09:00.250 MSK LOG: duration: 0.000 ms
2015-12-15 12:09:00.250 MSK LOG: duration: 0.000 ms parse <unnamed>: insert into customer (name, position, salary, id) values ($1, $2, $3, $4)
2015-12-15 12:09:00.250 MSK LOG: duration: 0.000 ms bind <unnamed>: insert into customer (name, position, salary, id) values ($1, $2, $3, $4)
2015-12-15 12:09:00.251 MSK DETAIL: parameters: $1 = 'Customer3', $2 = 'Position 3', $3 = '30', $4 = '7df32327-f2f5-4011-848d-55aafb3f09fa'
2015-12-15 12:09:00.251 MSK LOG: execute <unnamed>: insert into customer (name, position, salary, id) values ($1, $2, $3, $4)
2015-12-15 12:09:00.251 MSK DETAIL: parameters: $1 = 'Customer3', $2 = 'Position 3', $3 = '30', $4 = '7df32327-f2f5-4011-848d-55aafb3f09fa'
2015-12-15 12:09:00.251 MSK LOG: duration: 0.000 ms
2015-12-15 12:09:00.251 MSK LOG: duration: 0.000 ms parse S_1: insert into customer (name, position, salary, id) values ($1, $2, $3, $4)
2015-12-15 12:09:00.251 MSK LOG: duration: 0.000 ms bind S_1: insert into customer (name, position, salary, id) values ($1, $2, $3, $4)
2015-12-15 12:09:00.251 MSK DETAIL: parameters: $1 = 'Customer4', $2 = 'Position 4', $3 = '40', $4 = '1e55ab6a-8780-4c8f-8af2-2886d954f819'
2015-12-15 12:09:00.251 MSK LOG: execute S_1: insert into customer (name, position, salary, id) values ($1, $2, $3, $4)
2015-12-15 12:09:00.251 MSK DETAIL: parameters: $1 = 'Customer4', $2 = 'Position 4', $3 = '40', $4 = '1e55ab6a-8780-4c8f-8af2-2886d954f819'
2015-12-15 12:09:00.251 MSK LOG: duration: 0.000 ms
... the rest of the logs is identical and do not provide any valuable information...
因此，兩者之間的唯一區別是，當一批被禁用，Postgres將採取點點更多的時間來認識到，它應該重新使用準備好的語句。
所以我決定運行性能測試來證實這一點。
我試圖在空數據庫中插入30.000條記錄。
禁用配料，需要334ms。
啓用批處理後，耗時4650ms！
然後我刪除了所有對entityManager.flush的調用（只留下了entityManager.clear），時間降到了320ms。我不知道爲什麼hibernate的教程建議使用flush。我想這只是一個錯誤。
所以我猜想底線是這樣的：批處理工作，但它沒有提供任何真正的好處（至少對於postgres）。此外，使用不當（如在教程中）可能導致可怕的，可怕的結果。使用您自己的風險，並始終衡量真正的性能提高。
您是否確認JDBC驅動程序支持批量DDL操作？雖然數據庫可能支持它，但如果您的驅動程序版本不支持，它不會有任何影響。 – Naros
是的，jdbc驅動程序確實支持批處理。 – Alexey
添加您的'客戶'類。你正在使用Spring引導，所以你的'spring.jpa.properties'是你應該使用的，其他的幾乎沒用。 –