Spring數據倉庫多線程性能

我遇到了多個線程訪問同一個彈簧數據存儲庫對象的性能問題。許多線程在等待資源庫對象的鎖時被阻塞。大多數線程都在存儲庫對象上執行相同的查詢。當這個線程塊運行時，CPU在所有內核上都被最大化。偶爾它會下降，我認爲這是來自等待鎖存器對象的阻塞線程。我通過分析驗證了多個線程正在等待調用存儲庫對象中的相同方法。通過改變使用返回列表的方法的方法，我確實看到了性能的提升。但鎖定仍然是一個瓶頸。Spring數據倉庫多線程性能

更新：經過更多的研究，我得出結論，存儲庫對象是一個單身人士。這個對象在每個線程訪問它時都被鎖定。我如何建立資源庫對象的原型？（我會爲這個用例創建一個只讀存儲庫。）配置是否需要更改？ Spring數據已經做到了嗎？

MWE：

public interface EntityJpaRepository extends JpaRepository<Entity, Integer> { 
    @Query(value = "select * from SomeTable where id = (?1);", nativeQuery = true) 
    Entity findById(int id); 

    //Method that returns a list of Entities 
    @Query(value = "select * from SomeTable where id in (?1);", nativeQuery = true) 
    List<Entity> findAllWithIds(List<Integer> ids); 
} 

@Component 
@Scope("prototype") 
public class AThread implements Runnable { 
    @Autowired 
    EntityJpaRepository myRepository; 

    final int someId;   

    public AThread(int someId) { 
     this.someId = someId; 
    } 

    @Override 
    public void run() { 
     //may call subMethod 1 
     myRepository.findById(someId); 
     //may call subMethod 1 
     List<Integer> ids = someMethodWhichReturnsIDs(); 
     myRepository.findAllWithIds(ids); 
     //may call subMethod 1 
    } 

    public void subMethod1(){ 
     //sometimes loop 
     subMethod2(); 
     //may call 
    } 
    public void subMethod2(){ 
     //more stuff 
     List<Integer> ids = someMethodWhichReturnsIDs(); 
     //more stuff 
    } 
} 

public static void main(String[] args){ 
ThreadPoolTaskExecutor taskExecutor = (ThreadPoolTaskExecutor) ctx.getBean("taskExecutor"); 

List<int> someInts;//assume this is full of ints. 
for(int someId: someInts){ 
    taskExecutor.execute((Runnable)ctx.getBean("AThread", someId)); 
} 
waitThreads(taskExecutor);

我會說我得到的性能公平位出了什麼我現在有。我也不確定我是否已正確設置配置以使用多個線程/連接訪問數據庫。我不認爲這是問題，但我提供了完整的配置。歡迎任何性能提示。

@Configuration 
@EnableJpaRepositories(basePackages= {"org.repository"}) 
@ContextConfiguration(locations={"classpath:META-INF/spring/app-context.xml"}) 
@ComponentScan(basePackages = "org.somepackage") 
public class JpaConfiguration { 

    @Value("#{mainDataSource}") 
    private javax.sql.DataSource dataSource; 

    @Bean 
    public Map<String, Object> jpaProperties() { 
     Map<String, Object> props = new HashMap<String, Object>(); 
     props.put("hibernate.dialect", MySQLDialect.class.getName()); 
     props.put("hibernate.format_sql", true); 
     return props; 
    } 

    @Bean 
    public JpaVendorAdapter jpaVendorAdapter() { 
     HibernateJpaVendorAdapter hibernateJpaVendorAdapter = new HibernateJpaVendorAdapter(); 
     hibernateJpaVendorAdapter.setDatabase(Database.MYSQL); 
     return hibernateJpaVendorAdapter; 
    } 

    @Bean 
    public PlatformTransactionManager transactionManager() { 
     return new JpaTransactionManager(entityManagerFactory().getObject()); 
    } 

    @Bean 
    public LocalContainerEntityManagerFactoryBean entityManagerFactory() { 
     LocalContainerEntityManagerFactoryBean lef = new LocalContainerEntityManagerFactoryBean(); 
     lef.setDataSource(this.dataSource); 
     lef.setJpaPropertyMap(this.jpaProperties()); 
     lef.setJpaVendorAdapter(this.jpaVendorAdapter()); 
     return lef; 
    } 

    @Bean 
    public ThreadPoolTaskExecutor taskExecutor() { 
     ThreadPoolTaskExecutor pool = new ThreadPoolTaskExecutor(); 
     pool.setCorePoolSize(100); 
     pool.setMaxPoolSize(500); 
     pool.setWaitForTasksToCompleteOnShutdown(true); 
     pool.setKeepAliveSeconds(1800); 
     return pool; 
    } 
}

這是一個線程的堆棧跟蹤，當我暫停進程/應用程序時，它被阻塞。我也有pro-filer的樣本輸出。當它運行更長時間時，阻塞的時間加起來。它顯然被@Autowired存儲庫對象上的另一個線程阻塞。我以爲我通過使用原型範圍避免了這一點。

taskExecutor-1 BLOCKED 4.68003 6320 34 2062 1 [email protected] taskExecutor-9 


java.lang.ClassLoader.loadClass(Unknown Source) 
sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source) 
java.lang.ClassLoader.loadClass(Unknown Source) 
org.springframework.util.ClassUtils.isVisible(ClassUtils.java:1209) 
org.springframework.util.ClassUtils.getAllInterfacesForClassAsSet(ClassUtils.java:1136) 
org.springframework.util.ClassUtils.getAllInterfacesForClassAsSet(ClassUtils.java:1143) 
org.springframework.util.ClassUtils.getAllInterfacesForClass(ClassUtils.java:1099) 
org.springframework.orm.jpa.SharedEntityManagerCreator$SharedEntityManagerInvocationHandler.invoke(SharedEntityManagerCreator.java:302) 
com.sun.proxy.$Proxy41.createNativeQuery(Unknown Source) 
org.springframework.data.jpa.repository.query.NativeJpaQuery.createJpaQuery(NativeJpaQuery.java:65) 
org.springframework.data.jpa.repository.query.AbstractStringBasedJpaQuery.doCreateQuery(AbstractStringBasedJpaQuery.java:72) 
org.springframework.data.jpa.repository.query.AbstractJpaQuery.createQuery(AbstractJpaQuery.java:165) 
org.springframework.data.jpa.repository.query.JpaQueryExecution$SingleEntityExecution.doExecute(JpaQueryExecution.java:197) 
org.springframework.data.jpa.repository.query.JpaQueryExecution.execute(JpaQueryExecution.java:74) 
org.springframework.data.jpa.repository.query.AbstractJpaQuery.doExecute(AbstractJpaQuery.java:98) 
org.springframework.data.jpa.repository.query.AbstractJpaQuery.execute(AbstractJpaQuery.java:89) 
org.springframework.data.repository.core.support.RepositoryFactorySupport$QueryExecutorMethodInterceptor.doInvoke(RepositoryFactorySupport.java:421) 
org.springframework.data.repository.core.support.RepositoryFactorySupport$QueryExecutorMethodInterceptor.invoke(RepositoryFactorySupport.java:381) 
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) 
org.springframework.data.repository.core.support.RepositoryFactorySupport$DefaultMethodInvokingMethodInterceptor.invoke(RepositoryFactorySupport.java:512) 
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) 
org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:99) 
org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:281) 
org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96) 
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) 
org.springframework.dao.support.PersistenceExceptionTranslationInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:136) 
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) 
org.springframework.data.jpa.repository.support.CrudMethodMetadataPostProcessor$CrudMethodMetadataPopulatingMethodIntercceptor.invoke( CrudMethodMetadataPostProcessor.java:122) 
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) 
org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92) 
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) 
org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:213) 
com.sun.proxy.$Proxy81.findAllWithIds(Unknown Source) 
sun.reflect.GeneratedMethodAccessor64.invoke(Unknown Source) 
sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) 
java.lang.reflect.Method.invoke(Unknown Source) 
org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:333) 
org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:207) 
com.sun.proxy.$Proxy90.findAllWithIds(Unknown Source) 
org.somepackage.AThread.subMethod2(AThread.java:696) 
org.somepackage.AThread.subMethod1(AThread.java:346) 
org.somepackage.AThread.run(AThread.java:132) 
java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) 
java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) 
java.lang.Thread.run(Unknown Source)

來源

2017-09-16 Joshua Winfrey

如何得到'EntityJpaRepository'在main（用'CTX。 getBean'）並用'new AThread（theEntityJpaRepository）'創建你的線程。也不能使用某種方法來取代'List '而不是？ – 2017-09-16 04:17:30

我已經使用列表方法來獲得一些性能改進。但我仍然看到相同的行爲。多個線程正在等待進入存儲庫方法。我想認爲@Scope（「prototype」）註釋會給我一個Repository對象的副本。我開始認爲@Autowired註釋是一個共享（單例）對象。 –

你的代碼看起來非常合理。它允許對數據庫執行多達100個併發請求。不過，我不明白爲什麼你的線程被阻止。如果run方法的實現僅對數據庫執行單一隻讀請求，並且不能被隔離級別低於可序列化的其他線程阻塞。你能否詳細解釋一下在運行方法的實際實現中，「線程在等待資源庫對象鎖定時被阻塞」的問題？ –

這看起來非常像反覆加載類查詢的問題。同一類的重複加載可能會導致類加載器鎖爭用過多。（這裏例如：https://plumbr.eu/blog/locked-threads/classloading-and-locking）

的sharedEntityManager在加載線302類 - 這將在ClassLoader的上線404碰上synchronized塊

來源

2017-09-19 09:23:43

問題原來是我的錯誤。我改變了程序的優先級，仍然使用它運行的機器來完成其他任務。當我將任務的優先級改回正常時，問題就消失了。當數據庫進程和有關程序都處於同一優先級時，它也不存在。 –

Spring數據倉庫多線程性能

回答

相關問題