哈希內存使用率高於instagram測試結果

大家好我是redis新手，我正在關注instagram工程博客以達到優化目的。我通過哈希（每個1000個密鑰有1000個哈希）測試了1千萬密鑰存儲的內存使用情況。根據Instagram的帖子here它只需要16MB的存儲空間，但是我的測試佔用了38MB。任何人都可以告訴我我哪裏出錯了？哈希內存使用率高於instagram測試結果

這裏是我的測試代碼：

# -*- coding: utf-8 -*- 
import redis 
#pool=redis.ConnectionPool(host=127.0.0.1,port=6379,db=4) 
NUM_ENTRIES=1000000 
MAX_VAL=12000000 
def createData(min,max,userId): 
     r_server=redis.Redis(host='localhost',port=6379,db=5) 
     p=r_server.pipeline() 
     for i in xrange(0,1000): 
       for j in xrange(0,1000): 
         p.hset('follower:%s' % (i),j,j) 
     p.execute() 
     size = int(r_server.info()['used_memory']) 
     print '%s bytes, %s MB' % (size, size/1024/1024)

Redis的信息：

# Server 
redis_version:2.8.9 
redis_git_sha1:00000000 
redis_git_dirty:0 
redis_build_id:a9b5dff7da49156c 
redis_mode:standalone 
os:Linux 3.19.0-15-generic x86_64 
arch_bits:64 
multiplexing_api:epoll 
gcc_version:4.9.2 
process_id:11037 
run_id:c069c22be15f6b7cbd6490cea6d4ca497d8ad7cb 
tcp_port:6379 
uptime_in_seconds:230666 
uptime_in_days:2 
hz:10 
lru_clock:8643496 
config_file: 

# Clients 
connected_clients:1 
client_longest_output_list:0 
client_biggest_input_buf:0 
blocked_clients:0 

# Memory 
used_memory:41186920 
used_memory_human:39.28M 
used_memory_rss:60039168 
used_memory_peak:256243984 
used_memory_peak_human:244.37M 
used_memory_lua:33792 
mem_fragmentation_ratio:1.46 
mem_allocator:jemalloc-3.2.0 

# Persistence 
loading:0 
rdb_changes_since_last_save:0 
rdb_bgsave_in_progress:0 
rdb_last_save_time:1434659507 
rdb_last_bgsave_status:ok 
rdb_last_bgsave_time_sec:0 
rdb_current_bgsave_time_sec:-1 
aof_enabled:0 
aof_rewrite_in_progress:0 
aof_rewrite_scheduled:0 
aof_last_rewrite_time_sec:-1 
aof_current_rewrite_time_sec:-1 
aof_last_bgrewrite_status:ok 
aof_last_write_status:ok 

# Stats 
total_connections_received:21 
total_commands_processed:3010067 
instantaneous_ops_per_sec:0 
rejected_connections:0 
sync_full:0 
sync_partial_ok:0 
sync_partial_err:0 
expired_keys:0 
evicted_keys:0 
keyspace_hits:10 
keyspace_misses:0 
pubsub_channels:0 
pubsub_patterns:0 
latest_fork_usec:2774 

# Replication 
role:master 
connected_slaves:0 
master_repl_offset:0 
repl_backlog_active:0 
repl_backlog_size:1048576 
repl_backlog_first_byte_offset:0 
repl_backlog_histlen:0 

# CPU 
used_cpu_sys:264.43 
used_cpu_user:110.01 
used_cpu_sys_children:0.27 
used_cpu_user_children:1.55 

# Keyspace 
db5:keys=1000,expires=0,avg_ttl=0

來源

2015-06-18 riya ahuja

你還可以提供數據庫的輸出'INFO內存'以及配置指令'hash-max-ziplist-entries'和'hash-max-ziplist-value'？ –

@ItamarHaber hash-max-ziplist-entries 512和 hash-max-ziplist-value 64 –

這可能是由於你的服務器使用的hash-max-ziplist-entries默認設置，因爲你店1000場 - 這裏有一個小測試我跑你的代碼片段：

[email protected]:/tmp$ redis-cli config get hash-max-ziplist-entries 
1) "hash-max-ziplist-entries" 
2) "512" 
[email protected]:/tmp$ time python so.py 
56791944 bytes, 54 MB 

real 0m23.225s 
user 0m18.574s 
sys 0m0.377s 
[email protected]:/tmp$ redis-cli config set hash-max-ziplist-entries 1000 
OK 
[email protected]:/tmp$ redis-cli flushall 
OK 
[email protected]:/tmp$ time python so.py 
9112080 bytes, 8 MB 

real 0m28.928s 
user 0m18.663s 
sys 0m0.315s

來源

2015-06-19 14:43:35

.Thanx它解決了我的問題。我只有question.does它有任何缺點如果我增加hash-max-ziplist-entries值？ –

只是通常的空間時間折衷（RAM vs CPU）:) –

哈希內存使用率高於instagram測試結果

回答

相關問題