Couchbase在Java和.Net之間的memcached實現中的數據共享

Question

我試圖在Java和.Net之間共享存儲在couchbase memcached存儲桶中的數據。

我能夠讀取.Net中的Java中的字符串集，但每當我嘗試讀取Java中的.Net中設置的字符串時，結果都爲空。

所以有可能在一個couchbase服務器的memcache桶中的.Net和Java之間交換數據。

Answer 1

是的，這是可能的。在Java客戶端的情況下，它有一個內置的「代碼轉換器」，它將處理將java.lang.String轉換爲具有適當編碼的字節（UTF-8，我認爲我必須檢查）。 .NET方面將能夠讀取此數據。

事情變得粘滯的地方是每個客戶端庫如何存儲字符串。在memcached協議中，建議但不是必需的方法是使用標誌。面臨的挑戰是，每個客戶端庫的標誌都是不同的，Couchbase客戶端庫開發人員希望將其解析爲一組通用標誌。

這意味着，現在爲了標準化數據在兩個客戶端庫之間的存儲方式，您可能必須以特定方式設置客戶端庫。例如，Java具有可自定義的代碼轉換器，您可以擴展其中一個現有的代碼轉換器，以使用.NET客戶端庫使用的標誌讀取/寫入字符串。

讓我知道你正在使用什麼客戶端庫，我會用一個例子來更新它。

Answer 2

感謝您的回覆，我明白了。

.NET能夠讀取在Java中設置的字符串的原因是因爲enyimMemcached庫將緩存項目解釋爲字符串，如果它不識別該標誌。

因此，爲了能夠讀取Java中的字符串，我通過擴展Sp​​yObject並將其設置爲遠離它來忽略該標誌，從而簡單地創建了自己的自定義代碼轉換器。然後我通過自定義轉碼器與我的GET調用這個樣子，

_obj = GetMemcachedClient().get(key, new StringTranscoder()) 

我StringTranscoder類看起來是這樣的，

/** 
* Copyright (C) 2006-2009 Dustin Sallings 
* 
* Permission is hereby granted, free of charge, to any person obtaining a copy 
* of this software and associated documentation files (the "Software"), to deal 
* in the Software without restriction, including without limitation the rights 
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 
* copies of the Software, and to permit persons to whom the Software is 
* furnished to do so, subject to the following conditions: 
* 
* The above copyright notice and this permission notice shall be included in 
* all copies or substantial portions of the Software. 
* 
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING 
* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALING 
* IN THE SOFTWARE. 
*/ 

package cachingjavatestapp; 

import java.io.IOException; 
import net.spy.memcached.CachedData; 
import net.spy.memcached.compat.SpyObject; 
import net.spy.memcached.transcoders.Transcoder; 
import net.spy.memcached.transcoders.TranscoderUtils; 

/** 
* Transcoder that serializes and unserializes longs. 
*/ 
public final class StringTranscoder extends SpyObject implements 
    Transcoder<String> { 

    private static final int FLAGS = 0; 

    public boolean asyncDecode(CachedData d) { 
    return false; 
    } 

    public CachedData encode(java.lang.String l) { 
     try{ 
      return new CachedData(FLAGS, l.getBytes("UTF-8"), getMaxSize()); 
     } 
     catch (Exception e){ 
      return null; 
     } 
    } 


    public String decode(CachedData d) { 
     try{ 
      return new String(d.getData(), "UTF-8"); 
     }catch(Exception e){ 
      return null; 
     } 
    } 

    public int getMaxSize() { 
    return CachedData.MAX_SIZE; 
    } 
} 

爲了能夠.NET和Java之間交換數據。我只是使用json.net庫和gson庫來序列化對象，並將json字符串傳遞給memcached，然後將其作爲字符串拾取，然後使用json庫進行反序列化。

問候，

Answer 3

我知道這是一個相當古老的問題，但我有同樣的問題，我想我會分享我目前的解決方案。以下EnyimMemcached轉碼器與spymemcached中序列化/標記類型的方式更加匹配。顯然，如果你試圖在.Net和Java之間序列化對象，這是行不通的。但它可以讓你使用的不僅僅是字符串。

https://github.com/mikeleedev/EnyimMemcached/blob/master/Enyim.Caching/Memcached/Transcoders/SpymemcachedTranscoder.cs

public class SpymemcachedTranscoder : ITranscoder 
    { 
     #region Private Members 
     // General flags 
     private const uint SERIALIZED = 1; //00000000 00000001 
     private const uint COMPRESSED = 2; //00000000 00000010 <-- TODO - add support for compression 
     private const uint NOFLAG = 0; //00000000 00000000 

     // Special flags for specially handled types. 
     private const uint SPECIAL_MASK = 0xff00; //11111111 00000000 
     private const uint SPECIAL_BOOLEAN = (1 << 8); //00000001 00000000 
     private const uint SPECIAL_INT = (2 << 8); //00000010 00000000 
     private const uint SPECIAL_LONG = (3 << 8); //00000011 00000000 
     private const uint SPECIAL_DATE = (4 << 8); //00000100 00000000 
     private const uint SPECIAL_BYTE = (5 << 8); //00000101 00000000 
     private const uint SPECIAL_FLOAT = (6 << 8); //00000110 00000000 
     private const uint SPECIAL_DOUBLE = (7 << 8); //00000111 00000000 
     private const uint SPECIAL_BYTEARRAY = (8 << 8); //00001000 00000000 

     private readonly ArraySegment<byte> NullArray = new ArraySegment<byte>(new byte[0]); 
     private readonly DateTime EPOCH_START_DATETIME = new DateTime(1970, 1, 1, 0, 0, 0, DateTimeKind.Utc); 
     private readonly SpyMemcachedTranscoderUtils _spyTranscoderUtil = new SpyMemcachedTranscoderUtils(true); 
     #endregion 

     #region Serialize/Deserialize 
     CacheItem ITranscoder.Serialize(object value) 
     { 
      return this.Serialize(value); 
     } 

     object ITranscoder.Deserialize(CacheItem item) 
     { 
      return this.Deserialize(item); 
     } 

     protected virtual CacheItem Serialize(object value) 
     { 
      // raw data is a special case when some1 passes in a buffer (byte[] or ArraySegment<byte>) 
      if (value is ArraySegment<byte>) 
      { 
       // ArraySegment<byte> is only passed in when a part of buffer is being 
       // serialized, usually from a MemoryStream (To avoid duplicating arrays 
       // the byte[] returned by MemoryStream.GetBuffer is placed into an ArraySegment.) 
       return new CacheItem(SPECIAL_BYTEARRAY, (ArraySegment<byte>)value); 
      } 

      var tmpByteArray = value as byte[]; 

      // - or we just received a byte[]. No further processing is needed. 
      if (tmpByteArray != null) 
      { 
       return new CacheItem(SPECIAL_BYTEARRAY, new ArraySegment<byte>(tmpByteArray)); 
      } 

      uint flags = NOFLAG; 
      ArraySegment<byte> data; 
      TypeCode code = value == null ? TypeCode.Empty : Type.GetTypeCode(value.GetType()); 

      switch (code) 
      { 
       case TypeCode.Empty: 
       case TypeCode.DBNull: 
        flags = SPECIAL_BYTEARRAY; 
        data = this.SerializeNull(); 
        break; 
       case TypeCode.String: 
        flags = NOFLAG; 
        data = this.SerializeString((String)value); 
        break; 
       case TypeCode.Int64: 
        flags = SPECIAL_LONG; 
        data = this.SerializeInt64((Int64)value); 
        break; 
       case TypeCode.Int32: 
        flags = SPECIAL_INT; 
        data = this.SerializeInt32((Int32)value); 
        break; 
       case TypeCode.Boolean: 
        flags = SPECIAL_BOOLEAN; 
        data = this.SerializeBoolean((Boolean)value); 
        break; 
       case TypeCode.DateTime: 
        flags = SPECIAL_DATE; 
        data = this.SerializeDateTime((DateTime)value); 
        break; 
       case TypeCode.Byte: 
        flags = SPECIAL_BYTE; 
        data = this.SerializeByte((byte)value); 
        break; 
       case TypeCode.Single: //float 
        flags = SPECIAL_FLOAT; 
        data = this.SerializeSingle((float)value); 
        break; 
       case TypeCode.Double: 
        flags = SPECIAL_DOUBLE; 
        data = this.SerializeDouble((double)value); 
        break; 
       default: 
        flags = SERIALIZED; 
        data = this.SerializeObject(value); 
        break; 
      } 

      //TODO - determine when to apply compression and do it 

      return new CacheItem(flags, data); 
     } 

     protected virtual object Deserialize(CacheItem item) 
     { 
      if (item.Data.Array == null) 
       return null; 

      byte[] data = new byte[item.Data.Count]; 
      Array.Copy(item.Data.Array, item.Data.Offset, data, 0, item.Data.Count); 

      //TODO - compression support 
      //if ((item.Flags & COMPRESSED) != 0) 
      //{ 
      // data = Decompress(item.Data); 
      //} 

      if ((item.Flags & SERIALIZED) != 0) 
      { 
       return DeserializeObject(data); 
      } 

      uint flags = item.Flags & SPECIAL_MASK; 
      if (flags == NOFLAG) 
      { 
       return DeserializeString(data); 
      } 
      else 
      { 
       switch (flags) 
       { 
        case SPECIAL_BYTEARRAY: 
         return data; 
        case SPECIAL_BOOLEAN: 
         return this.DeserializeBoolean(data); 
        case SPECIAL_INT: 
         return this.DeserializeInt32(data); 
        case SPECIAL_LONG: 
         return this.DeserializeInt64(data); 
        case SPECIAL_DATE: 
         return this.DeserializeDateTime(data); 
        case SPECIAL_BYTE: 
         return this.DeserializeByte(data); 
        case SPECIAL_FLOAT: 
         return this.DeserializeSingle(data); 
        case SPECIAL_DOUBLE: 
         return this.DeserializeDouble(data); 
        default: 
         throw new InvalidOperationException(string.Format("SpyTranscoder undecodable with flags: {0}", flags)); 
       } 
      } 
     }  
     #endregion 

     #region Typed Serialization 

     protected virtual ArraySegment<byte> SerializeNull() 
     { 
      return NullArray; 
     } 

     protected virtual ArraySegment<byte> SerializeString(string value) 
     { 
      return new ArraySegment<byte>(Encoding.UTF8.GetBytes((string)value)); 
     } 

     protected virtual ArraySegment<byte> SerializeByte(byte value) 
     { 
      return new ArraySegment<byte>(_spyTranscoderUtil.EncodeByte(value)); 
     } 

     protected virtual ArraySegment<byte> SerializeBoolean(bool value) 
     { 
      return new ArraySegment<byte>(_spyTranscoderUtil.EncodeBoolean(value)); 
     } 

     protected virtual ArraySegment<byte> SerializeInt32(Int32 value) 
     { 
      return new ArraySegment<byte>(_spyTranscoderUtil.EncodeInt(value)); 
     } 

     protected virtual ArraySegment<byte> SerializeInt64(Int64 value) 
     { 
      return new ArraySegment<byte>(_spyTranscoderUtil.EncodeLong(value)); 
     } 

     protected virtual ArraySegment<byte> SerializeDateTime(DateTime value) 
     { 
      var epochMilliseconds = (long)(value - EPOCH_START_DATETIME).TotalMilliseconds; 
      return new ArraySegment<byte>(_spyTranscoderUtil.EncodeLong(epochMilliseconds)); 
     } 

     protected virtual ArraySegment<byte> SerializeDouble(Double value) 
     { 
      return new ArraySegment<byte>(_spyTranscoderUtil.EncodeLong(BitConverter.DoubleToInt64Bits(value))); 
     } 

     protected virtual ArraySegment<byte> SerializeSingle(Single value) 
     { 
      return new ArraySegment<byte>(_spyTranscoderUtil.EncodeLong(BitConverter.ToInt32(BitConverter.GetBytes(value), 0))); 
     } 

     protected virtual ArraySegment<byte> SerializeObject(object value) 
     { 
      using (var ms = new MemoryStream()) 
      { 
       new BinaryFormatter().Serialize(ms, value); 

       return new ArraySegment<byte>(ms.GetBuffer(), 0, (int)ms.Length); 
      } 
     } 

     #endregion 

     #region Typed deserialization 

     protected virtual String DeserializeString(byte[] value) 
     { 
      //return Encoding.UTF8.GetString(value.Array, value.Offset, value.Count); 
      return Encoding.UTF8.GetString(value); 
     } 

     protected virtual Boolean DeserializeBoolean(byte[] value) 
     { 
      return _spyTranscoderUtil.DecodeBoolean(value); 
     } 

     protected virtual Int32 DeserializeInt32(byte[] value) 
     { 
      return _spyTranscoderUtil.DecodeInt(value); 
     } 

     protected virtual Int64 DeserializeInt64(byte[] value) 
     { 
      return _spyTranscoderUtil.DecodeLong(value); 
     } 

     protected virtual DateTime DeserializeDateTime(byte[] value) 
     { 
      var epochMilliseconds = _spyTranscoderUtil.DecodeLong(value); 
      return EPOCH_START_DATETIME.AddMilliseconds(epochMilliseconds); 
     } 

     protected virtual Double DeserializeDouble(byte[] value) 
     { 
      return BitConverter.Int64BitsToDouble(_spyTranscoderUtil.DecodeLong(value)); 
     } 

     protected virtual Single DeserializeSingle(byte[] value) 
     { 
      byte[] bytes = BitConverter.GetBytes(_spyTranscoderUtil.DecodeInt(value)); 
      return BitConverter.ToSingle(bytes, 0); 
     } 

     protected virtual Byte DeserializeByte(byte[] data) 
     { 
      return _spyTranscoderUtil.DecodeByte(data); 
     } 

     protected virtual object DeserializeObject(byte[] value) 
     { 
      //using (var ms = new MemoryStream(value.Array, value.Offset, value.Count)) 
      using (var ms = new MemoryStream(value)) 
      { 
       return new BinaryFormatter().Deserialize(ms); 
      } 
     } 

     #endregion 

     #region GZip 
     private ArraySegment<byte> Compress(ArraySegment<byte> data) 
     { 
      using (var outStream = new MemoryStream()) 
      { 
       using (var compressStream = new GZipStream(outStream, CompressionMode.Compress)) 
       { 
        using (var inStream = new MemoryStream(data.Array)) 
        { 
         inStream.CopyTo(compressStream); 
         return new ArraySegment<byte>(outStream.ToArray()); 
        } 
       } 
      } 
     } 
     private ArraySegment<byte> Decompress(ArraySegment<byte> data) 
     { 
      using (var inStream = new MemoryStream(data.Array)) 
      { 
       using (var decompressStream = new GZipStream(inStream, CompressionMode.Decompress)) 
       { 
        using (var outStream = new MemoryStream()) 
        { 
         decompressStream.CopyTo(outStream); 
         return new ArraySegment<byte>(outStream.ToArray()); 
        } 
       } 
      } 
     } 
     #endregion 
    } 

    internal class SpyMemcachedTranscoderUtils 
    { 

     private readonly bool _packZeros; 

     public SpyMemcachedTranscoderUtils(bool pack = true) 
     { 
      _packZeros = pack; 
     } 

     public byte[] EncodeNum(long value, int maxBytes) 
     { 
      byte[] rv = new byte[maxBytes]; 
      for (int i = 0; i < rv.Length; i++) 
      { 
       int pos = rv.Length - i - 1; 
       rv[pos] = (byte)((value >> (8 * i)) & 0xff); 
      } 
      if (_packZeros) 
      { 
       int firstNon0 = 0; 
       // Just looking for what we can reduce 
       while (firstNon0 < rv.Length && rv[firstNon0] == 0) 
       { 
        firstNon0++; 
       } 
       if (firstNon0 > 0) 
       { 
        byte[] tmp = new byte[rv.Length - firstNon0]; 
        Array.Copy(rv, firstNon0, tmp, 0, rv.Length - firstNon0); 
        rv = tmp; 
       } 
      } 
      return rv; 
     } 

     public byte[] EncodeLong(long value) 
     { 
      return EncodeNum(value, 8); 
     } 

     public long DecodeLong(byte[] value) 
     { 
      long rv = 0; 
      foreach (byte i in value) 
      { 
       rv = (rv << 8) | (i < 0 ? 256 + i : i); 
      } 
      return rv; 
     } 

     public byte[] EncodeInt(int value) 
     { 
      return EncodeNum(value, 4); 
     } 

     public int DecodeInt(byte[] value) 
     { 
      if (value.Length > 4) 
       throw new InvalidOperationException("Too long to be an int (" + value.Length + ") bytes"); 

      return (int)DecodeLong(value); 
     } 

     public byte[] EncodeByte(byte value) 
     { 
      return new byte[] { value }; 
     } 

     public byte DecodeByte(byte[] value) 
     { 
      if (value.Length > 1) 
       throw new InvalidOperationException("Too long for a byte"); 

      byte rv = 0; 
      if (value.Length == 1) 
      { 
       rv = value[0]; 
      } 
      return rv; 
     } 

     public byte[] EncodeBoolean(bool b) 
     { 
      byte[] rv = new byte[1]; 
      rv[0] = (byte)(b ? '1' : '0'); 
      return rv; 
     } 

     public bool DecodeBoolean(byte[] value) 
     { 
      if (value.Length != 1) 
       throw new InvalidOperationException("Wrong length for a boolean"); 

      return value[0] == '1'; 
     } 
    }