2014-08-29 113 views
7

我無法連接到在Windows上運行Docker的HBase(banno/hbase-standalone圖像)。不過,我可以連接到本地安裝的HBase。連接到在Docker中運行的HBase

docker run -d -p 2181:2181 -p 60000:60000 -p 60010:60010 -p 60020:60020 -p 60030:60030 banno/hbase-standalone 

我還設置了對boot2docker-vm端口轉發(在Windows上運行時是必需的): enter image description here

我可以成功地遠程登錄到所有這些

banno/hbase-standalone圖像是使用運行我的本地主機上的端口。

接下來,這裏是我們在我們的測試中使用的代碼示例:

Configuration config = HBaseConfiguration.create(); 
config.clear(); 
config.setInt("timeout", 12000); 
config.set("zookeeper.znode.parent", "/hbase"); 
config.set("hbase.zookeeper.quorum", "127.0.0.1"); 
config.set("hbase.zookeeper.property.clientPort", "2181"); 
config.set("hbase.master", "127.0.0.1:60000"); 

final Configuration configuration = HBaseConfiguration.create(config); 

JobDefinition.Buildable.dumpProperties(configuration, newArrayList("hbase.*")); 

HBaseAdmin.checkHBaseAvailable(config); 

這將導致以下異常

Exception in thread "main" org.apache.hadoop.hbase.MasterNotRunningException: com.google.protobuf.ServiceException: java.net.UnknownHostException: unknown host: a3e6c240af20 
    at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$StubMaker.makeStub(HConnectionManager.java:1651) 
    at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$MasterServiceStubMaker.makeStub(HConnectionManager.java:1677) 
    at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getKeepAliveMasterService(HConnectionManager.java:1885) 
    at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.isMasterRunning(HConnectionManager.java:900) 
    at org.apache.hadoop.hbase.client.HBaseAdmin.checkHBaseAvailable(HBaseAdmin.java:2366) 
    at com.xxx.compute.hadoop.jobs.transaction.OurTest.main(OurTest.java:24) 
Caused by: com.google.protobuf.ServiceException: java.net.UnknownHostException: unknown host: a3e6c240af20 
    at org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1674) 
    at org.apache.hadoop.hbase.ipc.RpcClient$BlockingRpcChannelImplementation.callBlockingMethod(RpcClient.java:1715) 
    at org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$BlockingStub.isMasterRunning(MasterProtos.java:42561) 
    at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$MasterServiceStubMaker.isMasterRunning(HConnectionManager.java:1688) 
    at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$StubMaker.makeStubNoRetries(HConnectionManager.java:1597) 
    at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$StubMaker.makeStub(HConnectionManager.java:1623) 
    ... 5 more 
Caused by: java.net.UnknownHostException: unknown host: a3e6c240af20 
    at org.apache.hadoop.hbase.ipc.RpcClient$Connection.<init>(RpcClient.java:386) 
    at org.apache.hadoop.hbase.ipc.RpcClient.createConnection(RpcClient.java:352) 
    at org.apache.hadoop.hbase.ipc.RpcClient.getConnection(RpcClient.java:1526) 
    at org.apache.hadoop.hbase.ipc.RpcClient.call(RpcClient.java:1438) 
    at org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1657) 
    ... 10 more 

這是可以解釋的。我們運行Windows,這需要使用NAT運行boot2docker-vm虛擬機。同樣使用NAT的圖像的Docker容器在boot2docker-vm內運行。但是,由於Docker容器導出端口,端口對主機運行測試是「可見的」,並且端口被主機轉發。該名a3e6c240af20實際上來自多克爾容器ID,所以大概a3e6c240af20是泊塢窗容器主機名:

CONTAINER ID  IMAGE       COMMAND    CREATED    STATUS    PORTS                               NAMES 
a3e6c240af20  banno/hbase-standalone:latest "/bin/sh -c '/opt/hb 24 minutes ago  Up 24 minutes  0.0.0.0:2181->2181/tcp, 0.0.0.0:60000->60000/tcp, 0.0.0.0:60010->60010/tcp, 0.0.0.0:60020->60020/tcp, 0.0.0.0:60030->60030/tcp agitated_wozniak 

我不知道究竟是如何HBase的溝通工作,但顯然它使RPC調用的實例。 HBase Docker返回它的主機名,希望客戶端可以在那裏調用它。但由於boot2docker-vm和使用NAT運行的Docker容器,主機沒有看到Docker容器。

我嘗試添加a3e6c240af20到我的hosts文件:

127.0.0.1 a3e6c240af20 

然後我得到一個不同的錯誤,也是在RPC調用,這實際上並不能幫助我很多:

Exception in thread "main" org.apache.hadoop.hbase.MasterNotRunningException: com.google.protobuf.ServiceException: java.lang.NullPointerException 
    at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$StubMaker.makeStub(HConnectionManager.java:1651) 
    at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$MasterServiceStubMaker.makeStub(HConnectionManager.java:1677) 
    at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getKeepAliveMasterService(HConnectionManager.java:1885) 
    at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.isMasterRunning(HConnectionManager.java:900) 
    at org.apache.hadoop.hbase.client.HBaseAdmin.checkHBaseAvailable(HBaseAdmin.java:2366) 
    at com.xxx.compute.hadoop.jobs.transaction.OurTest.main(OurTest.java:24) 
Caused by: com.google.protobuf.ServiceException: java.lang.NullPointerException 
    at org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1674) 
    at org.apache.hadoop.hbase.ipc.RpcClient$BlockingRpcChannelImplementation.callBlockingMethod(RpcClient.java:1715) 
    at org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$BlockingStub.isMasterRunning(MasterProtos.java:42561) 
    at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$MasterServiceStubMaker.isMasterRunning(HConnectionManager.java:1688) 
    at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$StubMaker.makeStubNoRetries(HConnectionManager.java:1597) 
    at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$StubMaker.makeStub(HConnectionManager.java:1623) 
    ... 5 more 
Caused by: java.lang.NullPointerException 
    at org.apache.hadoop.hbase.ipc.RpcClient$Connection.writeRequest(RpcClient.java:1051) 
    at org.apache.hadoop.hbase.ipc.RpcClient.call(RpcClient.java:1440) 
    at org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1657) 
    ... 10 more 

不任何人都有一個建議如何解決這個問題?

+0

也許HBase的隨機打開的端口用於RPC請求,但碼頭工人容器不公開他們。我現在有同樣的問題。 – tobe 2014-11-21 02:57:57

回答

1

嘗試添加[boot2docker IP] a3e6c240af20而不是127.0.0.1因爲HBase的Java客戶端需要達到你的docker's host不完全localhost到達動物園管理員(CMIIW)。不確定它是否可以正常工作,但它可以在我的Windows中運行。

0

我使用oddpoet/hbase-cdh5泊塢窗圖像來避免此問題。

docker run -d -p 2181:2181 -p 60000:60000 -p 60010:60010 -p 60020:60020 -p 60030:60030 -h hbase oddpoet/hbase-cdh5 

fig.yml

hbase: 
    image: oddpoet/hbase-cdh5 
    hostname: hbase 
    ports: 
    - "3181:2181" 
    - "60000:60000" 
    - "60010:60010" 
    - "60020:60020" 
    - "60030:60030" 

我的配置文件

conf.set("hbase.zookeeper.quorum", zkPath); 
    conf.set("hbase.zookeeper.property.clientPort","2181"); 
    conf.set("zookeeper.znode.parent", "/hbase"); 

    conf.set("hbase.client.retries.number", "3"); // default 35 
    conf.set("hbase.rpc.timeout", "10000"); // default 60 secs 
    conf.set("hbase.rpc.shortoperation.timeout", "5000"); // default 10 secs