使用pymox嘲笑urllib2.urlopen和lxml.etree.parse

Question

我試圖測試一些使用urllib2和lxml的python代碼。

我看過幾篇博客文章和堆棧溢出帖子，其中人們想用urllib2測試引發的異常。我還沒有看到測試成功調用的例子。

我正走下正確的道路嗎？

有沒有人有建議讓這個工作？

這是我到目前爲止有：

import mox 
import urllib 
import urllib2 
import socket 
from lxml import etree 

# set up the test 
m = mox.Mox() 
response = m.CreateMock(urllib.addinfourl) 
response.fp = m.CreateMock(socket._fileobject) 
response.name = None # Needed because the file name is checked. 
response.fp.read().AndReturn("""<?xml version="1.0" encoding="utf-8"?> 
<foo>bar</foo>""") 
response.geturl().AndReturn("http://rss.slashdot.org/Slashdot/slashdot") 
response.read = response.fp.read # Needed since __init__ is not called on addinfourl. 
m.StubOutWithMock(urllib2, 'urlopen') 
urllib2.urlopen(mox.IgnoreArg(), timeout=10).AndReturn(response) 
m.ReplayAll() 

# code under test 
response2 = urllib2.urlopen("http://rss.slashdot.org/Slashdot/slashdot", timeout=10) 
# Note: response2.fp.read() and response2.read() do not behave the same, as defined above. 
# In [21]: response2.fp.read() 
# Out[21]: '<?xml version="1.0" encoding="utf-8"?>\n<foo>bar</foo>' 
# In [22]: response2.read() 
# Out[22]: <mox.MockMethod object at 0x97f326c> 
xcontent = etree.parse(response2) 

# verify test 
m.VerifyAll() 

它失敗：

Traceback (most recent call last): 
    File "/home/jon/mox_question.py", line 22, in <module> 
    xcontent = etree.parse(response2) 
    File "lxml.etree.pyx", line 2583, in lxml.etree.parse (src/lxml/lxml.etree.c:25057) 
    File "parser.pxi", line 1487, in lxml.etree._parseDocument (src/lxml/lxml.etree.c:63708) 
    File "parser.pxi", line 1517, in lxml.etree._parseFilelikeDocument (src/lxml/lxml.etree.c:63999) 
    File "parser.pxi", line 1400, in lxml.etree._parseDocFromFilelike (src/lxml/lxml.etree.c:62985) 
    File "parser.pxi", line 990, in lxml.etree._BaseParser._parseDocFromFilelike (src/lxml/lxml.etree.c:60508) 
    File "parser.pxi", line 542, in lxml.etree._ParserContext._handleParseResultDoc (src/lxml/lxml.etree.c:56659) 
    File "parser.pxi", line 624, in lxml.etree._handleParseResult (src/lxml/lxml.etree.c:57472) 
    File "lxml.etree.pyx", line 235, in lxml.etree._ExceptionContext._raise_if_stored (src/lxml/lxml.etree.c:6222) 
    File "parser.pxi", line 371, in lxml.etree.copyToBuffer (src/lxml/lxml.etree.c:55252) 
TypeError: reading from file-like objects must return byte strings or unicode strings 

這是因爲response.read（）不返回我期望它返回。

Answer 1

我根本不會鑽研urllib2的內部。這超出了你關心的範圍，我認爲。以下是使用StringIO執行此操作的簡單方法。這裏關鍵的一點是，你打算將XML解析爲鴨子類型時只需要像文件一樣，它不需要是一個實際的addinfourl實例。

import StringIO 
import mox 
import urllib2 
from lxml import etree 

# set up the test 
m = mox.Mox() 
response = StringIO.StringIO("""<?xml version="1.0" encoding="utf-8"?> 
<foo>bar</foo>""") 
m.StubOutWithMock(urllib2, 'urlopen') 
urllib2.urlopen(mox.IgnoreArg(), timeout=10).AndReturn(response) 
m.ReplayAll() 

# code under test 
response2 = urllib2.urlopen("http://rss.slashdot.org/Slashdot/slashdot", timeout=10) 
xcontent = etree.parse(response2) 

# verify test 
m.VerifyAll() 

Answer 2

它看起來像你的失敗與mox根本無關 - 導致錯誤的行是從response2讀取的，這是直接調用slashdot。也許檢查這個對象，看看它的內容是什麼？

編輯：我沒有看到上面的m.StubOutWithMock(urllib2, 'urlopen')行，所以我認爲你是比較兩個電話;一個嘲弄（迴應），一個不（迴應2）。更新後的答案如下。

Answer 3

呼應什麼彼得說，我只想補充一點，你可能不需要與lxml的內部比的urllib2的更多關注。通過嘲笑lxml.etree，你可以完全隔離你真正需要測試的代碼，你自己的代碼。下面是一個例子，它也展示瞭如何使用模擬對象來測試response.getcode（）調用。

import mox 
from lxml import etree 
import urllib2 

class TestRssDownload(mox.MoxTestBase): 

    def test_rss_download(self): 
     expected_response = self.mox.CreateMockAnything() 
     self.mox.StubOutWithMock(urllib2, 'urlopen') 
     self.mox.StubOutWithMock(etree, 'parse') 
     self.mox.StubOutWithMock(etree, 'iterwalk') 
     title_elem = self.mox.CreateMock(etree._Element) 
     title_elem.text = 'some title' 

     # Set expectations 
     urllib2.urlopen("http://rss.slashdot.org/Slashdot/slashdot", timeout=10).AndReturn(expected_response) 
     expected_response.getcode().AndReturn(200) 
     etree.parse(expected_response).AndReturn('some parsed content') 
     etree.iterwalk('some parsed content', tag='{http://purl.org/rss/1.0/}title').AndReturn([('end', title_elem),]) 

     # Code under test 
     self.mox.ReplayAll() 
     self.production_code() 

    def production_code(self): 
     response = urllib2.urlopen("http://rss.slashdot.org/Slashdot/slashdot", timeout=10) 
     response_code = response.getcode() 
     if 200 != response_code: 
      raise Exception('Houston, we have a problem ({0})'.format(response_code)) 
     tree = etree.parse(response) 
     for ev, elem in etree.iterwalk(tree, tag='{http://purl.org/rss/1.0/}title'): 
      # Do something with elem.text 
      print('{0}: {1}'.format(ev, elem.text))