使用postgresql創業板異步

Question

我正在使用Goliath（這是由eventmachine供電）和postgres寶石pg，目前我使用封鎖的方式pg寶石：conn.exec('SELECT * FROM products')（例如），我想知道是否有更好的方式連接到postgres數據庫？

Answer 1

pg庫提供對PostgreSQL的異步API的全面支持。我已經添加了如何用它來samples/目錄an example：

#!/usr/bin/env ruby 

require 'pg' 

# This is a example of how to use the asynchronous API to query the 
# server without blocking other threads. It's intentionally low-level; 
# if you hooked up the PGconn#socket to some kind of reactor, you 
# could make this much nicer. 

TIMEOUT = 5.0 # seconds to wait for an async operation to complete 
CONN_OPTS = { 
    :host  => 'localhost', 
    :dbname => 'test', 
    :user  => 'jrandom', 
    :password => 'banks!stealUR$', 
} 

# Print 'x' continuously to demonstrate that other threads aren't 
# blocked while waiting for the connection, for the query to be sent, 
# for results, etc. You might want to sleep inside the loop or 
# comment this out entirely for cleaner output. 
progress_thread = Thread.new { loop { print 'x' } } 

# Output progress messages 
def output_progress(msg) 
    puts "\n>>> #{msg}\n" 
end 

# Start the connection 
output_progress "Starting connection..." 
conn = PGconn.connect_start(CONN_OPTS) or 
    abort "Unable to create a new connection!" 
abort "Connection failed: %s" % [ conn.error_message ] if 
    conn.status == PGconn::CONNECTION_BAD 

# Now grab a reference to the underlying socket so we know when the 
# connection is established 
socket = IO.for_fd(conn.socket) 

# Track the progress of the connection, waiting for the socket to 
# become readable/writable before polling it 
poll_status = PGconn::PGRES_POLLING_WRITING 
until poll_status == PGconn::PGRES_POLLING_OK || 
     poll_status == PGconn::PGRES_POLLING_FAILED 

    # If the socket needs to read, wait 'til it becomes readable to 
    # poll again 
    case poll_status 
    when PGconn::PGRES_POLLING_READING 
     output_progress " waiting for socket to become readable" 
     select([socket], nil, nil, TIMEOUT) or 
      raise "Asynchronous connection timed out!" 

    # ...and the same for when the socket needs to write 
    when PGconn::PGRES_POLLING_WRITING 
     output_progress " waiting for socket to become writable" 
     select(nil, [socket], nil, TIMEOUT) or 
      raise "Asynchronous connection timed out!" 
    end 

    # Output a status message about the progress 
    case conn.status 
    when PGconn::CONNECTION_STARTED 
     output_progress " waiting for connection to be made." 
    when PGconn::CONNECTION_MADE 
     output_progress " connection OK; waiting to send." 
    when PGconn::CONNECTION_AWAITING_RESPONSE 
     output_progress " waiting for a response from the server." 
    when PGconn::CONNECTION_AUTH_OK 
     output_progress " received authentication; waiting for " + 
         "backend start-up to finish." 
    when PGconn::CONNECTION_SSL_STARTUP 
     output_progress " negotiating SSL encryption." 
    when PGconn::CONNECTION_SETENV 
     output_progress " negotiating environment-driven " + 
         "parameter settings." 
    end 

    # Check to see if it's finished or failed yet 
    poll_status = conn.connect_poll 
end 

abort "Connect failed: %s" % [ conn.error_message ] unless 
    conn.status == PGconn::CONNECTION_OK 

output_progress "Sending query" 
conn.send_query("SELECT * FROM pg_stat_activity") 

# Fetch results until there aren't any more 
loop do 
    output_progress " waiting for a response" 

    # Buffer any incoming data on the socket until a full result 
    # is ready. 
    conn.consume_input 
    while conn.is_busy 
     select([socket], nil, nil, TIMEOUT) or 
      raise "Timeout waiting for query response." 
     conn.consume_input 
    end 

    # Fetch the next result. If there isn't one, the query is 
    # finished 
    result = conn.get_result or break 

    puts "\n\nQuery result:\n%p\n" % [ result.values ] 
end 

output_progress "Done." 
conn.finish 

if defined?(progress_thread) 
    progress_thread.kill 
    progress_thread.join 
end 

我建議你讀的PQconnectStart功能和PostgreSQL的手冊Asynchronous Command Processing部分中的文檔，然後比較，與上面的樣品。

我之前沒有使用過EventMachine，但是如果它允許你註冊一個套接字並在它變得可讀寫時進行回調，那麼我認爲將數據庫調用集成到它中會相當容易。

我一直有意使用Ilya Grigorik's article on using Fibers to clean up evented code中的想法來使異步API更易於使用，但這是一個辦法。我確實有a ticket open跟蹤它，如果你有興趣/動機自己動手。

Answer 2

我不再對Pg非常熟悉，但我還沒有聽說任何流行的數據庫都可以異步連接。因此，在查詢期間，您仍然需要維護與數據庫的連接。因此，您仍然需要阻止某些位置的堆棧。

根據您的應用程序，您可能已經在儘可能地做到最好。

但是，當你正在處理某種輪詢應用程序（在相同的客戶端在短時間內發送大量請求），並且獲得響應更加重要，即使它是空的，那麼你可以寫一個紅寶石Fiber或者滔滔不絕的線程或流程，這些線程或流程長期存在並代理對數據庫的查詢並緩存結果。

例如：來自客戶端A的請求進入。Goliath應用程序使用一些唯一ID處理對數據庫進程的查詢，並用'沒有數據'來響應查詢。數據庫進程完成查詢並將結果保存到帶有ID的緩存中。當來自同一客戶端的下一個請求進入時，Goliath發現它已經有查詢結果在等待，從緩存中刪除結果並響應客戶端。同時，它會使用數據庫進程安排下一個查詢，以便儘早完成。如果下一個請求在最後一個請求完成之前進入，則不會安排新的查詢（不會增加查詢）。

通過這種方式，您的響應速度非常快且無阻塞，同時仍然能夠儘快從DB中提供新數據。當然，它們可能與實際數據有些不同步，但是，根據應用程序的不同，這可能不成問題。

Answer 3

這個想法是使用一個異步適配器到數據庫（Postgresql）連同一個已經連接的Web服務器（Goliath）來獲得性能。 Mike Perham去年爲Rails 2.3寫了一個PG activerecord adaptor。也許你可以使用它。

又如，Ilya Grigorik發佈了異步Rails堆棧的this demo。在這種情況下，偶數服務器是Thin，而數據庫是Mysql。安裝演示程序並嘗試使用和不使用EM感知驅動程序的基準測試。差異很大。

Answer 4

是的，你可以從goliath以非阻塞的方式訪問postgres。我有同樣的需要，並放在一起這個概念證明：https://github.com/levicook/goliath-postgres-spike