Clojure Cassandra驅動程序性能

Question

我正在測試6節點Cassandra羣集上Alia在Clojure上的性能。即使在多線程時，我只能獲得大約400次寫入/秒。通過使用Firebrand Cassandra驅動程序並在Java中手動處理線程，我們可以使用96個線程獲得5000次寫入/秒。

我在這裏使用代理時做錯了什麼？在正在運行的機器上，CPU使用率只有〜25％，似乎很低。

更新：在Alia建議的作者的指導下，利用準備好的語句而不是原始語句，以同步，單線程的方式實現了高達2500次/秒的增益。我仍然需要用Clojure進行多線程測試，並分別使用內置在Alia /底層Java驅動程序中的異步函數來查看哪個更快。

更新2：通過額外利用驅動程序內置的異步功能，我現在看到了與以下mpenet類似的結果。

(ns alia-perf-test.core 
    (:gen-class) 
    (:require [qbits.alia :as alia] 
      [qbits.hayt :as hayt])) 

(defn exec-query [session query] 
    (alia/execute session (hayt/->raw query))) 

(defmacro time-query 
    [expr] 
    `(let [start# (. System (nanoTime)) 
     ret# ~expr] 
    (/ (double (- (. System (nanoTime)) start#)) 1000000.0))) 

(defn write-entity 
    [total-time session entity] 
    (let [query (hayt/->raw (hayt/insert :entities (hayt/values entity) (hayt/using :timestamp 1234))) 
     query-time (time-query (alia/execute session query))] 
     (+ total-time query-time))) 

(defn generate-entity [] 
    {:id (str (java.util.UUID/randomUUID)) :num 0}) 

(defn write-something 
    [write-agent session] 
    (send-off write-agent write-entity 
     session 
     (generate-entity))) 

(defn -main [& args] 
    (let [cluster (alia/cluster ["server1" 
           "server2" 
           "server3" 
           "server4" 
           "server5" 
           "server6"] 
           :pooling-options {:core-connections-per-host [:local 16 :remote 16] 
               :max-connections-per-host [:local 1000 :remote 1000] 
               :max-simultaneous-requests-per-connection [:local 32 :remote 32] 
               :min-simultaneous-requests-per-connection [:local 16 :remote 16]}) 
     session (alia/connect cluster)] 
    (alia/set-consistency! :any) 
    (exec-query session (hayt/create-keyspace :aliaperftest 
               (hayt/with {:replication 
                  {:class "NetworkTopologyStrategy" 
                  :dc1 3 :dc2 3}}))) 
    (exec-query session (hayt/use-keyspace :aliaperftest)) 
    (exec-query session (hayt/create-table :entities 
              (hayt/column-definitions {:id :varchar 
                    :num :int 
                    :primary-key [:id]}))) 
    (let [num-entities 10000 
      write-agent (agent 0)] 
     (dotimes [n num-entities] 
     (write-something write-agent session)) 
     (await write-agent) 
     (println "Wrote" num-entities "entities in" @write-agent "ms -" 
      (* (/ num-entities @write-agent) 1000.0) "ops/sec")) 

    (exec-query session (hayt/drop-table :entities)) 
    (exec-query session (hayt/drop-keyspace :aliaperftest)) 
    (alia/shutdown session) 
    (alia/shutdown cluster) 
    (shutdown-agents))) 

Answer 1

更新：我有它做在異步模式下〜25秒鐘100K請求（使用特別的成功處理，有單節點），使用原子得到的結果，並等待所有響應到達。

通過使用自定義執行程序（取決於您的設置和配料），也可能獲得更多性能，但我沒有走得太遠。 請參閱：https://github.com/mpenet/alia/blob/master/docs/guide.md#executors