爲什麼二進制內存使用量成長上的RabbitMQ

Question

如何找到爲什麼二進制內存使用量成長（取決於連接數）

我用rabbitmqctl EVAL「[garbage_collect（P）|| P < - processes（）]。' 但統計

root@Broker1:~# /usr/sbin/rabbitmqctl status | grep binary  
    {binary,6709401976}, 

root@Broker1:~# free -m 
      total  used  free  shared buffers  cached 
Mem:   16084  4147  11937   22  161  1550 
-/+ buffers/cache:  2435  13649 
Swap:   4183   0  4183 

---

root@Broker2:~# /usr/sbin/rabbitmqctl status 


Status of node rabbit@Broker2 ... 
[{pid,515}, 
    {running_applications, 
[{rabbitmq_web_mqtt,"RabbitMQ MQTT-over-WebSockets adapter","3.6.9"}, 
    {rabbitmq_mqtt,"RabbitMQ MQTT Adapter","3.6.9"}, 
    {rabbitmq_web_stomp,"Rabbit WEB-STOMP - WebSockets to Stomp adapter", 
     "3.6.9"}, 
    {rabbitmq_stomp,"RabbitMQ STOMP plugin","3.6.9"}, 
    {rabbitmq_management,"RabbitMQ Management Console","3.6.9"}, 
    {rabbitmq_web_dispatch,"RabbitMQ Web Dispatcher","3.6.9"}, 
    {rabbitmq_management_agent,"RabbitMQ Management Agent","3.6.9"}, 
    {rabbit,"RabbitMQ","3.6.9"}, 
    {os_mon,"CPO CXC 138 46","2.3"}, 
    {cowboy,"Small, fast, modular HTTP server.","1.0.4"}, 
    {ranch,"Socket acceptor pool for TCP protocols.","1.3.0"}, 
    {ssl,"Erlang/OTP SSL application","5.3.6"}, 
    {public_key,"Public key infrastructure","0.22.1"}, 
    {cowlib,"Support library for manipulating Web protocols.","1.0.2"}, 
    {crypto,"CRYPTO","3.4.1"}, 
    {amqp_client,"RabbitMQ AMQP Client","3.6.9"}, 
    {rabbit_common, 
     "Modules shared by rabbitmq-server and rabbitmq-erlang-client", 
     "3.6.9"}, 
    {inets,"INETS CXC 138 49","5.10.3"}, 
    {mnesia,"MNESIA CXC 138 12","4.12.3"}, 
    {compiler,"ERTS CXC 138 10","5.0.2"}, 
    {sockjs,"SockJS","0.3.4"}, 
    {xmerl,"XML parser","1.3.7"}, 
    {syntax_tools,"Syntax tools","1.6.16"}, 
    {asn1,"The Erlang ASN1 compiler version 3.0.2","3.0.2"}, 
    {sasl,"SASL CXC 138 11","2.4.1"}, 
    {stdlib,"ERTS CXC 138 10","2.2"}, 
    {kernel,"ERTS CXC 138 10","3.0.3"}]}, 
{os,{unix,linux}}, 
{erlang_version, 
"Erlang/OTP 17 [erts-6.2] [source] [64-bit] [smp:10:10] [async-threads:160] [kernel-poll:true]\n"}, 
{memory, 
[{total,3874499056}, 
    {connection_readers,598416}, 
    {connection_writers,139368}, 
    {connection_channels,774752}, 
    {connection_other,176069624}, 
    {queue_procs,36524016}, 
    {queue_slave_procs,0}, 
    {plugins,80184008}, 
    {other_proc,0}, 
    {mnesia,48042392}, 
    {metrics,7245632}, 
    {mgmt_db,74517952}, 
    {msg_index,1751384}, 
    {other_ets,14801208}, 
    {binary,3379246384}, 
    {code,28040419}, 
    {atom,1000601}, 
    {other_system,33438764}]}, 
{alarms,[]}, 
{listeners, 
[{clustering,2000,"::"}, 
    {amqp,5672,"::"}, 
    {http,15672,"::"}, 
    {stomp,61613,"::"}, 
    {'http/web-stomp',15674,"::"}, 
    {mqtt,1883,"::"}, 
    {'http/web-mqtt',15675,"::"}]}, 
{vm_memory_high_watermark,{absolute,"12288MiB"}}, 
{vm_memory_limit,12884901888}, 
{disk_free_limit,50000000}, 
{disk_free,91027542016}, 
{file_descriptors, 
[{total_limit,65435}, 
    {total_used,9214}, 
    {sockets_limit,58889}, 
    {sockets_used,9002}]}, 
{processes,[{limit,1048576},{used,42726}]}, 
{run_queue,0}, 
{uptime,82689}, 
{kernel,{net_ticktime,60}} 
] 

---

   total  used  free  shared buffers  cached 
Mem:   16084  4517  11567   27  160  2487 
-/+ buffers/cache:  1869  14215 
Swap:   4189   0  4189 

我的記憶是免費的！但內存長大後連接到服務器非常緩慢！

集羣：兔@ BROKER1（其他城市）

的RabbitMQ 3.6.9，二郎17.3

rabbitmq.config

[ 
    {log_levels,  [{connection, none},{channel,none}]}, 
    {file_descriptors, [{total_limit, 65000}]}, 
    {kernel, [ 
      {log_levels,  [{connection, none},{channel,none}]}, 
      {inet_dist_listen_min, 2000}, 
      {inet_dist_listen_max, 65000} 
    ]}, 
    {rabbit, [ 
      {loopback_users, []}, 
      {vm_memory_high_watermark, {absolute, "12288MiB"}}, 
      {vm_memory_high_watermark_paging_ratio, 0.60}, 
      {tcp_listeners, [5672]}, 
      {log_levels,  [{connection, none},{channel,none}]}, 
      {heartbeat, 10000} 
    ]}, 
    {rabbitmq_mqtt, [ 
      {log_levels,  [{connection, none},{channel,none}]}, 
      {default_user,  <<"guest">>}, 
      {default_pass,  <<"guest">>}, 
      {allow_anonymous, false}, 
      {vhost,   <<"test">>}, 
      {exchange,   <<"amq.topic">>}, 
      {subscription_ttl, 180000}, 
      {prefetch,   10}, 
      {tcp_listeners, [1883]}, 
      {tcp_listen_options, [ 
        binary, 
        {packet, raw}, 
        {reuseaddr, false}, 
        {backlog, 8192}, 
        {nodelay, true} 
      ]} 
    ]}, 
    {rabbitmq_stomp, [ 
      {default_vhost, <<"test">>}, 
      {exchange,   <<"amq.topic">>}, 
      {tcp_listen_options, [ 
        {backlog, 128}, 
        {nodelay, true} 
      ]} 
    ]} 
]. 

Answer 1

我相信你的實現是不正確。我正在使用RabbitMQ在實時網站上進行實時通知，並且沒有任何問題。我有一個高達3500的併發客戶端，但一個8GB RAM的實例工作得不錯。

那你究竟要做什麼呢？

您還沒有指定您的用例。但是根據rabbitmq文檔，只有當你有很多未傳送的消息時，內存纔會增長。很多未傳送的消息可能會導致磁盤空間/ RAM空間取決於您的客戶端如何連接到您。

如果每個用戶有一個經紀人，那非常昂貴。相反，如果您的用戶要在多個端點（例如，電話和網絡）上同時收到相同的消息，請爲每個用戶交換一次。在這種情況下，請使用扇出交換並讓用戶訂閱它。當交易所中的信息達到零時，始終啓用自動刪除標誌。使用非持久性交換;但讓你的經紀人減少。

如果您確定只有一個設備上的用戶將連接到一臺設備，但不會超過此設備，則不需要扇出交換，而是爲每個用戶創建一個隊列。但是，啓用自動刪除標誌。

通過交換或隊列保持較少數量的代理和處理請求是有效的。

如果它適合您的用例，還有一個選擇是消息到期。假設您正在使用RabbitMQ向用戶發送通知。然而，您將它們寫入數據庫併發出通知，您可以通過RMQ發送它。如果用戶不在線，可能會持續約10分鐘或更長時間，屆滿是一個不錯的選擇。因爲它是實時消息傳遞，而不是持久消息傳遞。他將從數據庫中讀取它

注意：如果您解釋您的用例，我可以幫助您詳細說明。

閱讀了解更多：https://www.rabbitmq.com/blog/2014/10/30/understanding-memory-use-with-rabbitmq-3-4/

至於說到這裏，

所有二進制使用是由於在隊列中的消息。

最好的方法是使用非持久性隊列和自動過期消息！

注意，「內存」和「持久性」的消息在這裏不是反義詞： 的非持久性消息可以在內存壓力下被換出和 持久性消息可以在內存中了。

這就解釋了你的問題！