1. 首頁
  2. 問答
  3. 配置單元不允許冒充配置單元

配置單元不允許冒充配置單元

2016-11-15 104 views
2

我有一個配置了hadoop 2.7.2和配置單元2.1.0的hadoop羣集。配置單元不允許冒充配置單元

我用直線連接使用下列命令蜂巢：

beeline 
beeline> !connect jdbc:hive2://localhost:10000 
Enter username for jdbc:hive2://localhost:10000:

起初，鍵入任何用戶名可以成功連接到蜂巢。但是，在配置下列設置之後，會顯示錯誤。

//within hive-site.xml and hiveserver2-site.xml 
<property> 
<name>hive.server2.enable.doAs</name> 
<value>true</value> 
<description> 
    Setting this property to true will have HiveServer2 execute 
    Hive operations as the user making the calls to it. 
</description> 
</property> 

//within core-site.xml 
<property> 
<name>hadoop.proxyuser.hive.hosts</name> 
<value>*</value> 
</property> 

<property> 
    <name>hadoop.proxyuser.hive.groups</name> 
    <value>*</value> 
</property>

我已經重新啓動Hadoop集羣，但仍然出現以下消息：

Error: Failed to open new session: java.lang.RuntimeException: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): User: hive is not allowed to impersonate hive (state=,code=0)

調試輸出hiveserver2是如下：

16/11/15 11:28:46 [IPC Client (241742811) connection to /10.104.90.40:8020 from hive]: DEBUG ipc.Client: IPC Client (241742811) connection to /10.104.90.40:8020 from hive: starting, having connections 1 
16/11/15 11:28:46 [IPC Client (241742811) connection to /10.104.90.40:8020 from hive]: DEBUG ipc.Client: IPC Client (241742811) connection to /10.104.90.40:8020 from hive got value #-3 
16/11/15 11:28:46 [IPC Client (241742811) connection to /10.104.90.40:8020 from hive]: DEBUG ipc.Client: closing ipc connection to /10.104.90.40:8020: User: hive is not allowed to impersonate hive 
org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): User: hive is not allowed to impersonate hive 
     at org.apache.hadoop.ipc.Client.call(Client.java:1475) 
     at org.apache.hadoop.ipc.Client.call(Client.java:1412) 
     at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:229) 
     at com.sun.proxy.$Proxy27.getFileInfo(Unknown Source) 
     at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:771) 
     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) 
     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 
     at java.lang.reflect.Method.invoke(Method.java:606) 
     at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:191) 
     at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102) 
     at com.sun.proxy.$Proxy28.getFileInfo(Unknown Source) 
     at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:2108) 
     at org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:1305) 
     at org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:1301) 
     at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) 
     at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1301) 
     at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1424) 
     at org.apache.hadoop.hive.ql.session.SessionState.createRootHDFSDir(SessionState.java:674) 
     at org.apache.hadoop.hive.ql.session.SessionState.createSessionDirs(SessionState.java:622) 
     at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:550) 
     at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:513) 
     at org.apache.hive.service.cli.session.HiveSessionImpl.open(HiveSessionImpl.java:165) 
     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) 
     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 
     at java.lang.reflect.Method.invoke(Method.java:606) 
     at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:78) 
     at org.apache.hive.service.cli.session.HiveSessionProxy.access$000(HiveSessionProxy.java:36) 
     at org.apache.hive.service.cli.session.HiveSessionProxy$1.run(HiveSessionProxy.java:63) 
     at java.security.AccessController.doPrivileged(Native Method) 
     at javax.security.auth.Subject.doAs(Subject.java:415) 
     at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657) 
     at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:59) 
     at com.sun.proxy.$Proxy45.open(Unknown Source) 
     at org.apache.hive.service.cli.session.SessionManager.createSession(SessionManager.java:327) 
     at org.apache.hive.service.cli.session.SessionManager.openSession(SessionManager.java:279) 
     at org.apache.hive.service.cli.CLIService.openSessionWithImpersonation(CLIService.java:189) 
     at org.apache.hive.service.cli.thrift.ThriftCLIService.getSessionHandle(ThriftCLIService.java:414) 
     at org.apache.hive.service.cli.thrift.ThriftCLIService.OpenSession(ThriftCLIService.java:310) 
     at org.apache.hive.service.rpc.thrift.TCLIService$Processor$OpenSession.getResult(TCLIService.java:1377) 
     at org.apache.hive.service.rpc.thrift.TCLIService$Processor$OpenSession.getResult(TCLIService.java:1362) 
     at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) 
     at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) 
     at org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:56) 
     at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286) 
     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) 
     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) 
     at java.lang.Thread.run(Thread.java:745) 
16/11/15 11:28:46 [IPC Client (241742811) connection to /10.104.90.40:8020 from hive]: DEBUG ipc.Client: IPC Client (241742811) connection to /10.104.90.40:8020 from hive: closed 
16/11/15 11:28:46 [IPC Client (241742811) connection to /10.104.90.40:8020 from hive]: DEBUG ipc.Client: IPC Client (241742811) connection to /10.104.90.40:8020 from hive: stopped, remaining connections 0 
16/11/15 11:28:46 [HiveServer2-Handler-Pool: Thread-38]: WARN service.CompositeService: Failed to open session 
java.lang.RuntimeException: java.lang.RuntimeException: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): User: hive is not allowed to impersonate hive 
     at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:89) 
     at org.apache.hive.service.cli.session.HiveSessionProxy.access$000(HiveSessionProxy.java:36) 
     at org.apache.hive.service.cli.session.HiveSessionProxy$1.run(HiveSessionProxy.java:63) 
     at java.security.AccessController.doPrivileged(Native Method) 
     at javax.security.auth.Subject.doAs(Subject.java:415) 
     at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657) 
     at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:59) 
     at com.sun.proxy.$Proxy45.open(Unknown Source) 
     at org.apache.hive.service.cli.session.SessionManager.createSession(SessionManager.java:327) 
     at org.apache.hive.service.cli.session.SessionManager.openSession(SessionManager.java:279) 
     at org.apache.hive.service.cli.CLIService.openSessionWithImpersonation(CLIService.java:189) 
     at org.apache.hive.service.cli.thrift.ThriftCLIService.getSessionHandle(ThriftCLIService.java:414) 
     at org.apache.hive.service.cli.thrift.ThriftCLIService.OpenSession(ThriftCLIService.java:310) 
     at org.apache.hive.service.rpc.thrift.TCLIService$Processor$OpenSession.getResult(TCLIService.java:1377) 
     at org.apache.hive.service.rpc.thrift.TCLIService$Processor$OpenSession.getResult(TCLIService.java:1362) 
     at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) 
     at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) 
     at org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:56) 
     at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286) 
     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) 
     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) 
     at java.lang.Thread.run(Thread.java:745) 
Caused by: java.lang.RuntimeException: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): User: hive is not allowed to impersonate hive 
     at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:578) 
     at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:513) 
     at org.apache.hive.service.cli.session.HiveSessionImpl.open(HiveSessionImpl.java:165) 
     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) 
     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 
     at java.lang.reflect.Method.invoke(Method.java:606) 
     at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:78) 
     ... 21 more 
Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): User: hive is not allowed to impersonate hive 
     at org.apache.hadoop.ipc.Client.call(Client.java:1475) 
     at org.apache.hadoop.ipc.Client.call(Client.java:1412) 
     at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:229) 
     at com.sun.proxy.$Proxy27.getFileInfo(Unknown Source) 
     at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:771) 
     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) 
     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 
     at java.lang.reflect.Method.invoke(Method.java:606) 
     at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:191) 
     at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102) 
     at com.sun.proxy.$Proxy28.getFileInfo(Unknown Source) 
     at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:2108) 
     at org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:1305) 
     at org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:1301) 
     at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) 
     at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1301) 
     at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1424) 
     at org.apache.hadoop.hive.ql.session.SessionState.createRootHDFSDir(SessionState.java:674) 
     at org.apache.hadoop.hive.ql.session.SessionState.createSessionDirs(SessionState.java:622) 
     at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:550) 
     ... 28 more 
16/11/15 11:28:46 [HiveServer2-Handler-Pool: Thread-38]: DEBUG security.UserGroupInformation: PrivilegedAction as:hive (auth:PROXY) via hive (auth:SIMPLE) from:org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:59) 
16/11/15 11:28:46 [HiveServer2-Handler-Pool: Thread-38]: INFO session.SessionState: Updating thread name to 74dcfeae-04c1-494d-b491-df53f0d20039 HiveServer2-Handler-Pool: Thread-38 
16/11/15 11:28:46 [HiveServer2-Handler-Pool: Thread-38]: INFO session.SessionState: Resetting thread name to HiveServer2-Handler-Pool: Thread-38 
16/11/15 11:28:46 [HiveServer2-Handler-Pool: Thread-38]: DEBUG session.SessionState: Removing resource dir /tmp/hive_resources 
16/11/15 11:28:46 [HiveServer2-Handler-Pool: Thread-38]: INFO hive.metastore: Trying to connect to metastore with URI thrift://10.104.90.40:9083 
16/11/15 11:28:46 [HiveServer2-Handler-Pool: Thread-38]: INFO hive.metastore: Opened a connection to metastore, current connections: 3 
16/11/15 11:28:46 [HiveServer2-Handler-Pool: Thread-38]: INFO hive.metastore: Connected to metastore. 
16/11/15 11:28:46 [HiveServer2-Handler-Pool: Thread-38]: DEBUG metadata.Hive: Closing current thread's connection to Hive Metastore. 
16/11/15 11:28:46 [HiveServer2-Handler-Pool: Thread-38]: INFO hive.metastore: Closed a connection to metastore, current connections: 2 
16/11/15 11:28:46 [HiveServer2-Handler-Pool: Thread-38]: DEBUG ipc.Client: stopping client from cache: [email protected] 
16/11/15 11:28:47 [HiveServer2-Handler-Pool: Thread-38]: WARN thrift.ThriftCLIService: Error opening session: 
org.apache.hive.service.cli.HiveSQLException: Failed to open new session: java.lang.RuntimeException: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): User: hive is not allowed to impersonate hive 
     at org.apache.hive.service.cli.session.SessionManager.createSession(SessionManager.java:336) 
     at org.apache.hive.service.cli.session.SessionManager.openSession(SessionManager.java:279) 
     at org.apache.hive.service.cli.CLIService.openSessionWithImpersonation(CLIService.java:189) 
     at org.apache.hive.service.cli.thrift.ThriftCLIService.getSessionHandle(ThriftCLIService.java:414) 
     at org.apache.hive.service.cli.thrift.ThriftCLIService.OpenSession(ThriftCLIService.java:310) 
     at org.apache.hive.service.rpc.thrift.TCLIService$Processor$OpenSession.getResult(TCLIService.java:1377) 
     at org.apache.hive.service.rpc.thrift.TCLIService$Processor$OpenSession.getResult(TCLIService.java:1362) 
     at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) 
     at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) 
     at org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:56) 
     at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286) 
     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) 
     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) 
     at java.lang.Thread.run(Thread.java:745) 
Caused by: java.lang.RuntimeException: java.lang.RuntimeException: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): User: hive is not allowed to impersonate hive 
     at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:89) 
     at org.apache.hive.service.cli.session.HiveSessionProxy.access$000(HiveSessionProxy.java:36) 
     at org.apache.hive.service.cli.session.HiveSessionProxy$1.run(HiveSessionProxy.java:63) 
     at java.security.AccessController.doPrivileged(Native Method) 
     at javax.security.auth.Subject.doAs(Subject.java:415) 
     at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657) 
     at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:59) 
     at com.sun.proxy.$Proxy45.open(Unknown Source) 
     at org.apache.hive.service.cli.session.SessionManager.createSession(SessionManager.java:327) 
     ... 13 more 
Caused by: java.lang.RuntimeException: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): User: hive is not allowed to impersonate hive 
     at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:578) 
     at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:513) 
     at org.apache.hive.service.cli.session.HiveSessionImpl.open(HiveSessionImpl.java:165) 
     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) 
     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 
     at java.lang.reflect.Method.invoke(Method.java:606) 
     at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:78) 
     ... 21 more 
Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): User: hive is not allowed to impersonate hive 
     at org.apache.hadoop.ipc.Client.call(Client.java:1475) 
     at org.apache.hadoop.ipc.Client.call(Client.java:1412) 
     at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:229) 
     at com.sun.proxy.$Proxy27.getFileInfo(Unknown Source) 
     at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:771) 
     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) 
     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 
     at java.lang.reflect.Method.invoke(Method.java:606) 
     at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:191) 
     at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102) 
     at com.sun.proxy.$Proxy28.getFileInfo(Unknown Source) 
     at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:2108) 
     at org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:1305) 
     at org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:1301) 
     at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) 
     at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1301) 
     at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1424) 
     at org.apache.hadoop.hive.ql.session.SessionState.createRootHDFSDir(SessionState.java:674) 
     at org.apache.hadoop.hive.ql.session.SessionState.createSessionDirs(SessionState.java:622) 
     at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:550) 
     ... 28 more

我在網上搜索，發現大部分的解決方案與此錯誤相關的是actualLy我配置的設置導致問題。

有誰知道如何解決這個問題？

更新：

經過一番嘗試，我發現這個錯誤涉及到哪個用戶正在使用的啓動蜂巢服務。

在我有用戶配置單元之前啓動配置單元存儲Metastore和hiveserver2導致錯誤消息。

但是使用用戶hadoop這是啓動hadoop namenode來啓動hiveserver2的用戶擺脫錯誤消息。

我沒有弄清楚爲什麼使用不同的用戶是一個臨時解決方法。

來源

2016-11-15 Heyang Wang

+0

爲什麼你想用特定的用戶開始Metastore（爲什麼不只是hadoop？）。 - 只是猜測，但也許啓動Metastore的用戶定義了配置/日誌的放置位置。而對於一些用戶來說，這些位置可能不存在或可能不夠容易訪問。 –

+0

起初，這是因爲大多數這些在線教程都使用配置單元作爲用戶。但後來，我想可能是所有的蜂巢服務使用蜂巢服務是不必要的sice蜂巢用戶只具有蜂巢文件夾的特權，它不能搞亂hadoop或spark文件夾。 –

+0

也許我從您的評論中得出了錯誤的結論，但如果您擔心訪問錯誤的文件夾，我建議您只使用「標準」用戶帳戶進行初始設置，然後使用自定義用戶帳戶。 （像王和管理王）。 –

回答

0

我也遇到類似的問題。這是我爲了使它工作而做的。 在蜂巢-site.xml中，你有這樣的特性：

<property> 
     <name>hive.conf.restricted.list</name> 
     <value>hive.security.authenticator.manager,hive.security.authorization.manager,**hive.users.in.admin.role**</value> 
    </property>

我只是刪除了部分大膽上面所強調的：hive.users.in.admin.role。

它開始爲我工作。

來源

2017-02-08 07:22:41

相關問題

 相關問題