通過boost :: asio發送原始數據:: asio

Question

我試圖通過boost :: asio發送原始數據，因爲boost :: serialization對於我的需求來說太慢了。繼各種實例和提升文檔，我有一個客戶端：

SimulationClient：

void SimulationClient::sendData(std::vector<WaveformDefinition>waveformPackets) { 
     socket.async_send_to(boost::asio::buffer(waveformPackets), 
       receiver_endpoint, 
       boost::bind(&ClientEnvironmentEngine::sendComplete, this, 
         boost::asio::placeholders::error, 
         boost::asio::placeholders::bytes_transferred)); 
} 

我試圖坦納桑斯伯裏的解決方案下面，但無法得到它的工作。不過，我使用具有成功：

class WaveformReceiver { 
    WaveformDefinition *buffer; 

    WaveformReceiver(){ 
     buffer = new WaveformDefinition[MAX_WAVEFORMS]; 
     startReceive(); 
    } 

    void startReceive() { 
     socket_.async_receive_from(boost::asio::null_buffers(), remote_endpoint_, 
       boost::bind(&WaveformReceiver::handleReceive, this, 
       boost::asio::placeholders::error, 
       boost::asio::placeholders::bytes_transferred)); 
    } 

    void handleReceive(const boost::system::error_code& error, 
     std::size_t size/*bytes_transferred*/) 
    { 
      if (!error) 
      { 
       int available = socket_.available(); 
       int numWaveforms = available/sizeof(WaveformDefinition_c); 
       socket_.receive(boost::asio::buffer(buffer, available)); 

       //copy buffer into another buffer so we can re-use the original buffer for the next read 
       WaveformDefinition_c* tempBuffer = new WaveformDefinition_c[numWaveforms]; 
       std::memcpy (tempBuffer, buffer, available); 

       //schedule a thread to handle the array of waveforms that we copied 
       threadPool.schedule(boost::bind(handleWaveforms, tempBuffer, numWaveforms)); 
       //start listening for more waveforms 
       startReceive(); 
      } 
    } 
} 

坦納，或別人，你能告訴我，如果我在做什麼也應該工作，或者如果我剛開幸運，它目前正在？

Answer 1

問題的根本部分是關於序列化和反序列化集合。

在不控制服務器和客戶端的編譯器和體系結構的情況下，發送原始結構通常是不安全的，因爲字節表示在系統之間可能會有所不同。雖然編譯器和體系結構在這種特定情況下是相同的，但#pragma pack(1)是無關緊要的，因爲WAVEFORM_DATA_STRUCT沒有作爲原始內存寫入套接字。相反，爲收集write操作提供了多個內存緩衝區。

boost::array<boost::asio::mutable_buffer,2> buffer = {{ 
    boost::asio::buffer(&waveformPacket->numWaveforms, ...), // &numWaveforms 
    boost::asio::buffer(waveformPacket->waveforms)   // &waveforms[0] 
}}; 

有各種工具可以幫助序列化數據結構，例如Protocol Buffers。

---

下面的代碼將演示序列化網絡通信的數據結構的基礎知識。爲了簡化代碼和解釋，我選擇了專注於序列化和反序列化，而不是從套接字讀寫。位於本節下面的另一個示例將顯示更多的原始方法，它假定使用相同的編譯器和體系結構。

與鹼性foo類型開始：

struct foo 
{ 
    char a; 
    char b; 
    boost::uint16_t c; 
}; 

可以確定該數據可以被打包成4個字節總數。下面是一種可能的導線reprensetation：

0  8  16  24  32 
|--------+--------+--------+--------| 
| a | b |  c  | 
'--------+--------+--------+--------' 

隨着所確定的線表示，兩個功能可以用於序列化（保存）一個foo對象到緩衝器，而另一個可被用於從一個反序列化（負載）foo緩衝。由於foo.c大於一個字節，所以這些函數也需要考慮endianness。我選擇在Boost.Asio細節名稱空間中使用endian字節交換函數來實現某些平臺中立性。

/// @brief Serialize foo into a network-byte-order buffer. 
void serialize(const foo& foo, unsigned char* buffer) 
{ 
    buffer[0] = foo.a; 
    buffer[1] = foo.b; 

    // Handle endianness. 
    using ::boost::asio::detail::socket_ops::host_to_network_short; 
    boost::uint16_t c = host_to_network_short(foo.c); 
    std::memcpy(&buffer[2], &c, sizeof c); 
} 

/// @brief Deserialize foo from a network-byte-order buffer. 
void deserialize(foo& foo, const unsigned char* buffer) 
{ 
    foo.a = buffer[0]; 
    foo.b = buffer[1]; 

    // Handle endianness. 
    using ::boost::asio::detail::socket_ops::network_to_host_short; 
    boost::uint16_t c; 
    std::memcpy(&c, &buffer[2], sizeof c); 
    foo.c = network_to_host_short(c); 
} 

與序列化和反序列爲foo完成後，下一步就是要處理foo對象的集合。在編寫代碼之前，需要確定電線表示。在這種情況下，我決定在一個具有32位計數字段的foo元素序列前綴。

0  8  16  24  32 
|--------+--------+--------+--------| 
|  count of foo elements [n] | 
|--------+--------+--------+--------| 
|   serialized foo [0]  | 
|--------+--------+--------+--------| 
|   serialized foo [1]  | 
|--------+--------+--------+--------| 
|    ...    | 
|--------+--------+--------+--------| 
|   serialized foo [n-1]  | 
'--------+--------+--------+--------' 

再次，可以引入兩個輔助功能，序列化和反序列化foo對象的集合，並且還需要佔計數字段的字節順序。

/// @brief Serialize a collection of foos into a network-byte-order buffer. 
template <typename Foos> 
std::vector<unsigned char> serialize(const Foos& foos) 
{ 
    boost::uint32_t count = foos.size(); 

    // Allocate a buffer large enough to store: 
    // - Count of foo elements. 
    // - Each serialized foo object. 
    std::vector<unsigned char> buffer(
     sizeof count +   // count 
     foo_packed_size * count); // serialize foo objects 

    // Handle endianness for size. 
    using ::boost::asio::detail::socket_ops::host_to_network_long; 
    count = host_to_network_long(count); 

    // Pack size into buffer. 
    unsigned char* current = &buffer[0]; 
    std::memcpy(current, &count, sizeof count); 
    current += sizeof count; // Adjust position. 

    // Pack each foo into the buffer. 
    BOOST_FOREACH(const foo& foo, foos) 
    { 
    serialize(foo, current); 
    current += foo_packed_size; // Adjust position. 
    } 

    return buffer; 
}; 

/// @brief Deserialize a buffer into a collection of foo objects. 
std::vector<foo> deserialize(const std::vector<unsigned char>& buffer) 
{ 
    const unsigned char* current = &buffer[0]; 

    // Extract the count of elements from the buffer. 
    boost::uint32_t count; 
    std::memcpy(&count, current, sizeof count); 
    current += sizeof count; 

    // Handle endianness. 
    using ::boost::asio::detail::socket_ops::network_to_host_long; 
    count = network_to_host_long(count); 

    // With the count extracted, create the appropriate sized collection. 
    std::vector<foo> foos(count); 

    // Deserialize each foo from the buffer. 
    BOOST_FOREACH(foo& foo, foos) 
    { 
    deserialize(foo, current); 
    current += foo_packed_size; 
    } 

    return foos; 
}; 

下面是完整的示例代碼：

#include <iostream> 
#include <vector> 
#include <boost/asio.hpp> 
#include <boost/asio/detail/socket_ops.hpp> // endian functions 
#include <boost/cstdint.hpp> 
#include <boost/foreach.hpp> 
#include <boost/tuple/tuple.hpp>   // boost::tie 
#include <boost/tuple/tuple_comparison.hpp> // operator== for boost::tuple 

/// @brief Mockup type. 
struct foo 
{ 
    char a; 
    char b; 
    boost::uint16_t c; 
}; 

/// @brief Equality check for foo objects. 
bool operator==(const foo& lhs, const foo& rhs) 
{ 
    return boost::tie(lhs.a, lhs.b, lhs.c) == 
     boost::tie(rhs.a, rhs.b, rhs.c); 
} 

/// @brief Calculated byte packed size for foo. 
/// 
/// @note char + char + uint16 = 1 + 1 + 2 = 4 
static const std::size_t foo_packed_size = 4; 

/// @brief Serialize foo into a network-byte-order buffer. 
/// 
/// @detail Data is packed as follows: 
/// 
/// 0  8  16  24  32 
/// |--------+--------+--------+--------| 
/// | a | b |  c  | 
/// '--------+--------+--------+--------' 
void serialize(const foo& foo, unsigned char* buffer) 
{ 
    buffer[0] = foo.a; 
    buffer[1] = foo.b; 

    // Handle endianness. 
    using ::boost::asio::detail::socket_ops::host_to_network_short; 
    boost::uint16_t c = host_to_network_short(foo.c); 
    std::memcpy(&buffer[2], &c, sizeof c); 
} 

/// @brief Deserialize foo from a network-byte-order buffer. 
void deserialize(foo& foo, const unsigned char* buffer) 
{ 
    foo.a = buffer[0]; 
    foo.b = buffer[1]; 

    // Handle endianness. 
    using ::boost::asio::detail::socket_ops::network_to_host_short; 
    boost::uint16_t c; 
    std::memcpy(&c, &buffer[2], sizeof c); 
    foo.c = network_to_host_short(c); 
} 

/// @brief Serialize a collection of foos into a network-byte-order buffer. 
/// 
/// @detail Data is packed as follows: 
/// 
/// 0  8  16  24  32 
/// |--------+--------+--------+--------| 
/// |  count of foo elements [n] | 
/// |--------+--------+--------+--------| 
/// |   serialized foo [0]  | 
/// |--------+--------+--------+--------| 
/// |   serialized foo [1]  | 
/// |--------+--------+--------+--------| 
/// |    ...    | 
/// |--------+--------+--------+--------| 
/// |   serialized foo [n-1]  | 
/// '--------+--------+--------+--------' 
template <typename Foos> 
std::vector<unsigned char> serialize(const Foos& foos) 
{ 
    boost::uint32_t count = foos.size(); 

    // Allocate a buffer large enough to store: 
    // - Count of foo elements. 
    // - Each serialized foo object. 
    std::vector<unsigned char> buffer(
     sizeof count +   // count 
     foo_packed_size * count); // serialize foo objects 

    // Handle endianness for size. 
    using ::boost::asio::detail::socket_ops::host_to_network_long; 
    count = host_to_network_long(count); 

    // Pack size into buffer. 
    unsigned char* current = &buffer[0]; 
    std::memcpy(current, &count, sizeof count); 
    current += sizeof count; // Adjust position. 

    // Pack each foo into the buffer. 
    BOOST_FOREACH(const foo& foo, foos) 
    { 
    serialize(foo, current); 
    current += foo_packed_size; // Adjust position. 
    } 

    return buffer; 
}; 

/// @brief Deserialize a buffer into a collection of foo objects. 
std::vector<foo> deserialize(const std::vector<unsigned char>& buffer) 
{ 
    const unsigned char* current = &buffer[0]; 

    // Extract the count of elements from the buffer. 
    boost::uint32_t count; 
    std::memcpy(&count, current, sizeof count); 
    current += sizeof count; 

    // Handle endianness. 
    using ::boost::asio::detail::socket_ops::network_to_host_long; 
    count = network_to_host_long(count); 

    // With the count extracted, create the appropriate sized collection. 
    std::vector<foo> foos(count); 

    // Deserialize each foo from the buffer. 
    BOOST_FOREACH(foo& foo, foos) 
    { 
    deserialize(foo, current); 
    current += foo_packed_size; 
    } 

    return foos; 
}; 

int main() 
{ 
    // Create a collection of foo objects with pre populated data. 
    std::vector<foo> foos_expected(5); 
    char a = 'a', 
     b = 'A'; 
    boost::uint16_t c = 100; 

    // Populate each element. 
    BOOST_FOREACH(foo& foo, foos_expected) 
    { 
    foo.a = a++; 
    foo.b = b++; 
    foo.c = c++; 
    } 

    // Serialize the collection into a buffer. 
    std::vector<unsigned char> buffer = serialize(foos_expected); 

    // Deserialize the buffer back into a collection. 
    std::vector<foo> foos_actual = deserialize(buffer); 

    // Compare the two. 
    std::cout << (foos_expected == foos_actual) << std::endl; // expect 1 

    // Negative test. 
    foos_expected[0].c = 0; 
    std::cout << (foos_expected == foos_actual) << std::endl; // expect 0 
} 

其產生1和0了預期的效果。

---

如果使用相同的編譯器和體系結構，那麼它可能有可能從原始緩衝液作爲foo對象數組重新解釋foo對象的連續序列，以及與複製構造填充std::vector<foo>。例如：

// Create and populate a contiguous sequence of foo objects. 
std::vector<foo> foo1; 
populate(foo1); 

// Get a handle to the contiguous memory block. 
const char* buffer = reinterpret_cast<const char*>(&foo1[0]); 

// Populate a new vector via iterator constructor. 
const foo* begin = reinterpret_cast<const foo*>(buffer); 
std::vector<foo> foos2(begin, begin + foos1.size()); 

最後，foo1應等於foo2。 foo2中的foo對象將由位於foo1所有的內存中的重新解釋的foo對象複製構建。

#include <iostream> 
#include <vector> 
#include <boost/cstdint.hpp> 
#include <boost/foreach.hpp> 
#include <boost/tuple/tuple.hpp>   // boost::tie 
#include <boost/tuple/tuple_comparison.hpp> // operator== for boost::tuple 

/// @brief Mockup type. 
struct foo 
{ 
    char a; 
    char b; 
    boost::uint16_t c; 
}; 

/// @brief Equality check for foo objects. 
bool operator==(const foo& lhs, const foo& rhs) 
{ 
    return boost::tie(lhs.a, lhs.b, lhs.c) == 
     boost::tie(rhs.a, rhs.b, rhs.c); 
} 

int main() 
{ 
    // Create a collection of foo objects with pre populated data. 
    std::vector<foo> foos_expected(5); 
    char a = 'a', 
     b = 'A'; 
    boost::uint16_t c = 100; 

    // Populate each element. 
    BOOST_FOREACH(foo& foo, foos_expected) 
    { 
    foo.a = a++; 
    foo.b = b++; 
    foo.c = c++; 
    } 

    // Treat the collection as a raw buffer. 
    const char* buffer = 
     reinterpret_cast<const char*>(&foos_expected[0]); 

    // Populate a new vector. 
    const foo* begin = reinterpret_cast<const foo*>(buffer); 
    std::vector<foo> foos_actual(begin, begin + foos_expected.size()); 

    // Compare the two. 
    std::cout << (foos_expected == foos_actual) << std::endl; 

    // Negative test. 
    foos_expected[0].c = 0; 
    std::cout << (foos_expected == foos_actual) << std::endl; 
} 

如同其它方法，這產生的1和0預期的結果。

Answer 2

首先，它不安全使用pragma pack(1)。包裝可能與不同的編譯器/拱不同。此外，您將在協議更改中遇到問題。我建議改用google protobuf。

二。您正在發送std::vector，但此向量的實際數據不在結構內WAVEFORM_DATA_STRUCT（向量將其數據保存在堆中）。所以，你發送矢量和它的指向堆到另一臺機器，這個指針definetly無效。你需要以某種方式序列化你的向量。

P.S.與boost :: asio沒有任何關係，但這個問題是關於正確的序列化/反序列化。