由於縮放原因,我們最近從單個ActiveMQ代理切換到代理網絡。儘管絕大多數情況下一切正常,但我們在白天全新部署代理後遇到了一個奇怪的問題:使用代理網絡中的臨時隊列的請求/應答模式的ActiveMQ/Camel故障切換 - 無法發佈到已刪除的臨時隊列
首先,我們使用的是ActiveMQ 5.12.1和Camel 2.13。 4用於java方法和JMS端點之間的集成。代理端是使用以下配置
<broker useJmx="${activemq.expose.jmx}" persistent="false"
brokerName="${activemq.brokerName}" xmlns="http://activemq.apache.org/schema/core">
<sslContext>
<amq:sslContext keyStore="${activemq.broker.keyStore}"
keyStorePassword="${activemq.broker.keyStorePassword}"
trustStore="${activemq.broker.trustStore}"
trustStorePassword="${activemq.broker.trustStorePassword}" />
</sslContext>
<systemUsage>
<systemUsage>
<memoryUsage>
<memoryUsage limit="${activemq.memoryUsage}" />
</memoryUsage>
<tempUsage>
<tempUsage limit="${activemq.tempUsage}" />
</tempUsage>
</systemUsage>
</systemUsage>
<destinationPolicy>
<policyMap>
<policyEntries>
<policyEntry queue=">" enableAudit="false">
<networkBridgeFilterFactory>
<conditionalNetworkBridgeFilterFactory
replayWhenNoConsumers="true" />
</networkBridgeFilterFactory>
</policyEntry>
</policyEntries>
</policyMap>
</destinationPolicy>
<networkConnectors>
<networkConnector name="queues"
uri="static:(${activemq.otherBrokers})"
networkTTL="2" dynamicOnly="true"
decreaseNetworkConsumerPriority="true"
conduitSubscriptions="false">
<excludedDestinations>
<topic physicalName=">" />
</excludedDestinations>
</networkConnector>
<networkConnector name="topics"
uri="static:(${activemq.otherBrokers})"
networkTTL="1" dynamicOnly="true"
decreaseNetworkConsumerPriority="true"
conduitSubscriptions="true">
<excludedDestinations>
<queue physicalName=">" />
</excludedDestinations>
</networkConnector>
</networkConnectors>
<transportConnectors>
<transportConnector
uri="${activemq.protocol}${activemq.host}:${activemq.tcp.port}?needClientAuth=true"
updateClusterClients="true" rebalanceClusterClients="true" />
<transportConnector
uri="${activemq.websocket.protocol}${activemq.websocket.host}:${activemq.websocket.port}?needClientAuth=true"
updateClusterClients="true" rebalanceClusterClients="true" />
</transportConnectors>
</broker>
具有以下佔位符目前由3個成員的代理網絡值
activemq.tcp.port=9000
activemq.protocol=ssl://
activemq.brokerName=activemq-server1.com
activemq.expose.jmx=true
activemq.otherBrokers=ssl://server2.com:9000,ssl://server3.com:9000
activemq.websocket.port=9001
activemq.websocket.protocol=stomp+ssl://
activemq.websocket.host=server1.com
activemq.memoryUsage=1gb
activemq.tempUsage=1gb
在客戶端被用於下面的駱駝配置
<bean id="xxx.activemq.redeliveryPolicy" class="org.apache.activemq.RedeliveryPolicy">
<property name="maximumRedeliveries" value="0" />
</bean>
<bean id="xxx.activemq.jmsConnectionFactory" class="org.apache.activemq.ActiveMQSslConnectionFactory">
<property name="trustStore" value="${activemq.broker.trustStore}" />
<property name="trustStorePassword" value="${activemq.broker.trustStorePassword}" />
<property name="keyStore" value="${activemq.broker.keyStore}" />
<property name="keyStorePassword" value="${activemq.broker.keyStorePassword}" />
<property name="brokerURL" value="${activemq.broker.url}" />
<property name="redeliveryPolicy" ref="xxx.activemq.redeliveryPolicy" />
</bean>
<bean id="xxx.activemq.jmsConfiguration" class="org.apache.activemq.camel.component.ActiveMQConfiguration">
<property name="receiveTimeout" value="6000" />
<property name="connectionFactory" ref="xxx.activemq.pooledConnectionFactory" />
</bean>
<bean id="xxx.activemq.pooledConnectionFactory"
class="org.apache.activemq.pool.PooledConnectionFactory"
init-method="start" destroy-method="stop">
<property name="maxConnections" value="8" />
<property name="idleTimeout" value="0" />
<property name="timeBetweenExpirationCheckMillis"
value="10000" />
<property name="connectionFactory"
ref="xxx.activemq.jmsConnectionFactory" />
</bean>
<bean id="xxx.activemq.jms.abstractComponent" abstract="true"
class="org.apache.activemq.camel.component.ActiveMQComponent">
<property name="configuration"
ref="xxx.activemq.jmsConfiguration" />
<property name="connectionFactory"
ref="xxx.activemq.pooledConnectionFactory" />
<property name="allowNullBody" value="true" />
<property name="transferException" value="true" />
<property name="defaultTaskExecutorType"
value="#{T(org.apache.camel.component.jms.DefaultTaskExecutorType).ThreadPool}" />
<property name="requestTimeout" value="5000" />
</bean>
<bean id="xxx.activemq.jms.queue"
parent="xxx.activemq.jms.abstractComponent">
<property name="concurrentConsumers" value="2" />
<property name="maxConcurrentConsumers" value="2" />
</bean>
連接網址爲
activemq.broker.url=failover:(ssl://server1.com:9000,ssl://server2.com:9000,ssl://server3.com:9000)?randomize=true
請求/回覆EIP是通過讓生產者設置一個相應的jmsReplyTo頭並使駝峯默認使用臨時隊列的InOut來實現的。
在部署之前,所有消息傳遞都按預期工作,但是之後對於某些請求/應答隊列,我們將開始在生產者端獲得超時。
在製片方
Caused by: org.apache.camel.ExchangeTimedOutException: The OUT message was not received within: 5000 millis due reply message with correlationID: Camel-ID-xxx-intranet-phs-49404-1457684675710-8-11 not received on destination: temp-queue://ID:xxx.intranet.phs-41986-1457684806758-1:3:1.
Exchange[Message: BeanInvocation public abstract xxx.xxx.rapi.dto.RemoteDTO xxx.xxx.xxx.facade.RemoteFacade.findRemoteDTO(java.lang.String,java.lang.Long) with [xxx, 31333]]]
,並在消費者方面:
Caused by: javax.jms.InvalidDestinationException: Cannot publish to a deleted Destination: temp-queue://ID:xxx.intranet.phs-41986-1457684806758-1:3:1
我們已經從那以後做了一些研究,發現問題會出現下面的條目在日誌中出現了每當網絡的任意代理關閉時,然後只對那些在關機命中時有臨時隊列打開以供回覆並且故障轉移到新代理的生產者。之後這個問題會持續存在,直到他重新啓動。一旦他重新開始後重新開始,一切都恢復正常。該問題也在grokbase以及activemq-failover-with-temporary-queues-on-a-network-of-brokers和activemq-how-to-handle-broker-failovers-while-using-temporary-queues這兩個主題上進行了描述。我們已經嘗試了activemq-how-to-handle-broker-failovers-while-using-temporary-queues中給出的一個解決方案來設置緩存超時,但沒有從中獲得任何結果,另一個建議的爲客戶提供諮詢監聽的選項在我們的設置中並不是真正的選擇,因爲我們希望利用功能如clusterRebalancing,以便在運行時更容易將其他代理添加到網絡中。
我們還發現了一些關於駱駝和ActiveMQ方面的JIRA問題,如CAMEL-3193描述了這個問題,並且明顯地修復了我們更新的版本,所以我們很困惑。目前我們正在考慮通過臨時隊列切換到獨佔應答隊列來解決這個問題,但首先想問問我們是否可能在某處丟失了一些配置。
如果您需要任何附加信息,請直接詢問!