spring批處理Hadoop失敗，出現org.springframework.beans.factory.BeanNotOfRequiredTypeException

我有一個觸發hadoop作業的spring批處理作業。請看下面我的工作配置。當我啓動的工作，我得到下面的異常...spring批處理Hadoop失敗，出現org.springframework.beans.factory.BeanNotOfRequiredTypeException

配置：

<hdp:job id="mr-my-job" 
     input-path="/data/input/" 
     output-path="/data/output/" 
     jar-by-class="org.test.Main" 
     mapper="org.test.Test1$Map" 
     combiner="org.test.Test1$Combiner" 
     reducer="org.test.Test1$ReduceFromCombiner" 
     number-reducers="7" 
     scope="step" />

例外：

org.springframework.beans.factory.BeanNotOfRequiredTypeException: Bean named 'mr-my-job' must be of type [org.apache.hadoop.mapreduce.Job], but was actually of type [$Proxy5] 
     at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:375) 
     at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:199) 
     at org.springframework.data.hadoop.mapreduce.JobExecutor.findJobs(JobExecutor.java:254) 
     at org.springframework.data.hadoop.mapreduce.JobExecutor.startJobs(JobExecutor.java:166) 
     at org.springframework.data.hadoop.batch.mapreduce.JobTasklet.execute(JobTasklet.java:90) 
     at org.springframework.data.hadoop.batch.mapreduce.JobTasklet$$FastClassBySpringCGLIB$$4805a065.invoke(<generated>) 
     at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) 
     at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:717) 
     at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) 
     at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:133) 
     at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:121) 
     at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) 
     at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:653) 
     at org.springframework.data.hadoop.batch.mapreduce.JobTasklet$$EnhancerBySpringCGLIB$$3be21557.execute(<generated>) 
     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) 
     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) 
     at java.lang.reflect.Method.invoke(Method.java:597) 
     at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317) 
     at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190) 
     at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) 
     at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:133) 
     at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:121) 
     at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) 
     at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:207) 
     at $Proxy2.execute(Unknown Source) 
     at org.springframework.batch.core.step.tasklet.TaskletStep$ChunkTransactionCallback.doInTransaction(TaskletStep.java:406) 
     at org.springframework.batch.core.step.tasklet.TaskletStep$ChunkTransactionCallback.doInTransaction(TaskletStep.java:330)

我加入CGLIB 3.1到類路徑，並添加 <bean class="org.springframework.batch.core.scope.StepScope" p:proxyTargetClass="true" />在我的XML，但仍然有相同的錯誤。

我調試，發現異常在個JobExecutor類拋出下面線而變換型[email protected]的豆到[email protected]。我認爲CGLIB的自動代理在這裏沒有得到應用，因爲它仍在使用JDK代理。

個JobExecutor類：

for (String name : jobNames) { 
    js.add(beanFactory.getBean(name, Job.class)); 
}

AbstractBeanFactory類

return getTypeConverter().convertIfNecessary(bean, requiredType);

請指點。

來源

2016-01-04 user3222372

可能在某處（在Tasklet中，我想）你正在爲一個具體的Hadoop作業類而不是一個接口進行佈線。檢查你的代碼 –

感謝@LucaBassoRicci的迴應。我把我的工作配置放在原文中。你能檢查我在哪裏做錯了嗎？ – user3222372

從「my-mr-job」中刪除'scope =「step」' –

謝謝盧卡巴索利瑪竇。從「my-mr-job」中刪除範圍=「步驟」工作。

來源

2016-01-05 16:38:35 user3222372

spring批處理Hadoop失敗，出現org.springframework.beans.factory.BeanNotOfRequiredTypeException

回答

相關問題