2016-04-25 89 views
0

發生hbase客戶機批處理數據時發生錯誤。起初沒關係。一段時間後它錯了!詳細的錯誤是:請求的行超出HRegion上的doMiniBatchMutation的範圍

:1次,org.apache.hadoop.hbase.exceptions.FailedSanityCheckException:請求的行超出範圍doMiniBatchMutation上HRegion idcard,bfef6945ac273d83 \ X00 \ X00 \ X00 \ X00 \ X00 \ x17 \ xcc $,1461584032622.dadb8843fe441dac4a3d4d7669597ef5。,startKey ='bfef6945ac273d83 \ x00 \ x00 \ x00 \ x00 \ x00 \ x17 \ xCC $',getEndKey()='',row ='9a6ec957205e1d74 \ x00 \ x00 \ x00 \ x00 \ x01 \ x90 \ x1F \ xF5' at org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(RSRpcServices.java:712) at org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java :662) at org.apache.hadoop.hb org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:2046) at org.apache.hadoop.hb ase.protobuf.generated.ClientProtos $ ClientService $ 2.callBlockingMethod(ClientProtos.java:32393) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2117) at org.apache.hadoop.hbase .ipc.CallRunner.run(CallRunner.java:104) at org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:133) at org.apache.hadoop.hbase.ipc.RpcExecutor $ 1。運行(RpcExecutor.java:108) 在java.lang.Thread.run(Thread.java:745)

的環境是:

  1. HBase的HBase的-1.1.3
  2. hadoop2.6
  3. HBase的客戶端1.2.0

HBase的客戶端的代碼是:

public static void batchPutData(Connection connection, long startNum, long count) throws IOException, ParseException{ 
     //table 
     Table table = connection.getTable(TableName.valueOf(TABLE_NAME)); 
     //index table 
     Table index_table = connection.getTable(TableName.valueOf(INDEX_TABLE_NAME)); 

     //random name 
     RandomChineseName randomChineseName = new RandomChineseName(); 
     //random car 
     RandomCar randomCar = new RandomCar(); 

     List<Put> puts = new ArrayList<Put>(); 
     List<Put> indexPlateputs = new ArrayList<Put>(); 
     for(long i = 0; i < count; i++){ 
      long index = startNum+i; 

      Date birthdate = RandomUtils.randomDate(); 
      String birthdateStr = DateUtil.dateToStr(birthdate, "yyyy-MM-dd"); 
      boolean isBoy = i%2==0?true:false; 
      String name = isBoy?randomChineseName.randomBoyName():randomChineseName.randomGirlName(); 
      String nation = RandomUtils.randomNation(); 
      String plate = randomCar.randomPlate(); 

      byte[] idbuff = Bytes.toBytes(index); 
      String hashPrefix = MD5Hash.getMD5AsHex(idbuff).substring(0, 16); 

      //create a put for table 
      Put put = new Put(Bytes.add(Bytes.toBytes(hashPrefix), idbuff)); 
      put.addColumn(Bytes.toBytes("idcard"), Bytes.toBytes("name"), Bytes.toBytes(name)); 
      put.addColumn(Bytes.toBytes("idcard"), Bytes.toBytes("sex"), Bytes.toBytes(isBoy?1:0)); 
      put.addColumn(Bytes.toBytes("idcard"), Bytes.toBytes("birthdate"), Bytes.toBytes(birthdateStr)); 
      put.addColumn(Bytes.toBytes("idcard"), Bytes.toBytes("nation"), Bytes.toBytes(nation)); 
      put.addColumn(Bytes.toBytes("idcard"), Bytes.toBytes("plate"), Bytes.toBytes(plate)); 
      puts.add(put); 

      //create a put for index table 
      String namehashPrefix = MD5Hash.getMD5AsHex(Bytes.toBytes(name)).substring(0, 16); 
      byte[] bprf = Bytes.add(Bytes.toBytes(namehashPrefix), Bytes.toBytes(name)); 
      bprf = Bytes.add(bprf, Bytes.toBytes(SPLIT), Bytes.toBytes(birthdateStr)); 
      Put namePut = new Put(Bytes.add(bprf, Bytes.toBytes(SPLIT), Bytes.toBytes(index))); 
      namePut.addColumn(Bytes.toBytes("index"), Bytes.toBytes("idcard"), Bytes.toBytes(0)); 
      indexPlateputs.add(namePut); 

      //insert for every ten thousands 
      if(i%10000 == 0){ 
       table.put(puts); 
       index_table.put(indexPlateputs); 
       puts.clear(); 
       indexPlateputs.clear(); 
      } 
     } 
    } 
+0

你試過了什麼?產生這個錯誤的代碼是什麼? – Whitefret

+0

我通過hbase的客戶端每隔一萬次批量插入數據。 – Jeff

回答

1

好像與HBase的版本衝突。將HBase版本更改爲1.1.4或1.0.0或其他穩定版本以便嘗試。

+0

好,沒錯!謝謝! – Jeff