Jedis（Redis）減速

Question

我將很多文本插入到redis中以便逐行存儲頻率。但是，jedis/redis在執行一定操作後需要花費大量時間執行 e操作，並且程序以錯誤結束： java.lang.OutOfMemoryError。

這裏是我的測試主要文件： 公共類臨時{

private ExecutorService executor; 

public temp() { 
    executor = Executors.newFixedThreadPool(5); 
} 

public static void main(String[] args) { 
    temp ob = new temp(); 

    System.out.println("starting"); 
    for(long i =0;i<10000000;i++) { 
     if (i%10000 == 0) { 
      System.out.println(i); 
     } 
     String x = Integer.toString(new Random().nextInt()); 
     ob.executor.submit(new Runner1("abra"+x)); 
     ob.executor.submit(new Runner2("delhi"+x)); 
    } 

    try { 
     if (ob.executor.awaitTermination(Long.MAX_VALUE, TimeUnit.NANOSECONDS)) { 
      System.out.println("completed"); 
     } 
    } catch (Exception ex) { 
     ex.printStackTrace(); 
    } 

} 


} 

這裏是我的兩個參賽者： 亞軍1：

public class Runner1 implements Runnable { 
//private static RedisClientUtil redisClient = null; 
private String key; 
private static Integer count = 0; 

public Runner1(String key) { 
    this.key = key; 
} 

public void run() { 
    try { 

     ArrayList<ArrayList<Object>> cmd = new ArrayList<ArrayList<Object>>(); 

     String offer_title = this.key + " this is thread1"; 
     String offer_title_words[] = offer_title.split(" "); 
     for (String word : offer_title_words) { 
      // INCR the frequency in reddis 
      cmd.add(GenerateUtils.getArrayList("incrBy", "test"+word, 1)); 
     } 

     List<Object> responses = RedisbenchmarkTest.getLocalhostJedisPool().executePipelinedAndReturnResponses(0,cmd); 
     cmd = null; 
     responses = null; 

     updateNumberOfRowsInserted(); 

    } catch (Exception ex) { 
     ex.printStackTrace(); 
    } 
} 

    private synchronized void updateNumberOfRowsInserted() { 
     //logging 
     count++; 
     if(count%10000==0) 
      System.out.println("Thread 1 : " + count); 
    } 



} 

亞軍2：

public class Runner2 implements Runnable { 
//private static RedisClientUtil redisClient = null; 
private String key; 
private static Integer count = 0; 

public Runner2(String key) { 
    this.key = key; 
} 

public void run() { 
    try { 

     ArrayList<ArrayList<Object>> cmd = new ArrayList<ArrayList<Object>>(); 

     String offer_title = this.key + " this is thread2"; 
     String offer_title_words [] = offer_title.split(" "); 
     for (String word : offer_title_words) { 
      // INCR the category_word in reddis 
      cmd.add(GenerateUtils.getArrayList("incrBy","test1"+word,1)); 
     }     RedisbenchmarkTest.getLocalhostJedisPool().executePipelinedWithoutReturningResponses(0,cmd); 
     updateNumberOfRowsInserted(); 

    } catch (Exception ex) { 
     ex.printStackTrace(); 
    } 
} 

private synchronized void updateNumberOfRowsInserted() { 
    //logging 
    count++; 
    if(count%10000==0) 
     System.out.println("Thread 2 : " + count); 
} 

} 

這是我的Redis客戶端：

public class RedisbenchmarkTest { 

    private static RedisClientUtil localhostJedisPool; 


private static final JedisPoolConfig standardJedisPoolConfig = new JedisPoolConfig() {{ 
    setMaxTotal(500); 
    setMaxIdle(20); 
    setMaxWaitMillis(500); 
    setTestOnBorrow(false); 
    setTestOnReturn(false); 
}}; 

private static final int semiLowTimeout = 500; 

static { 
    initialize(); 
} 

public static void initialize() { 
    localhostJedisPool = new RedisClientUtil(
      standardJedisPoolConfig 
      , "localhost" 
      , 6379 
      , semiLowTimeout 
    ); 
    } 



    public static RedisClientUtil getLocalhostJedisPool() { 
     if (localhostJedisPool == null) { 
      initialize(); 
    } 
    return localhostJedisPool; 
    } 
    } 

Answer 1

• 你應該redis.conf禁用bgsave如果Redis的開始bgsave通常巨大滯後暗示，直到保存完成。
• 您應該確保redis.conf中的maxmemory足夠高。我猜你的實例正在運行。

Answer 2

Redis不是內存不足的那個，你的Java過程是。

您沒有發佈完整的實際錯誤，我不是一個Java的人，但Caused by: java.lang.OutOfMemoryError: Java heap space可能會爲您提供一些答案。