Python上下文管理器：有條件地執行正文？

Question

我正在寫一個基於MPI的應用程序（但MPI在我的問題中無關緊要，我只提到它只是爲了公開基本原理），在某些情況下，當工作項目少於進程時，我需要創建一個新的溝通者排除無關進程。最後，新的溝通者必須被有工作要做的流程（並且只能由他們來解放）釋放。

一個巧妙的辦法來做到這一點是寫：

with filter_comm(comm, nworkitems) as newcomm: 
    ... do work with communicator newcomm... 

身體僅由有工作要做的進程正在執行。

上下文管理器中有避免執行正文的方法嗎？我知道上下文管理者已經正確地被設計來避免隱藏控制流，但是我想知道是否有可能規避這種情況，因爲在我的情況下，我認爲這是清楚的理由。

Answer 1

有條件地跳過上下文管理身體的能力已經提出，並拒絕在PEP 377記錄。

以下是一些如何實現功能的方法。

首先什麼不起作用：不從上下文管理器產生。

@contextlib.contextmanager 
def drivercontext(): 
    driver, ok = driverfactory() 
    try: 
    if ok: 
     yield driver 
    else: 
     print 'skip because driver not ok' 
    finally: 
    driver.quit() 

with drivercontext() as driver: 
    dostuff(driver) 

沒有產生將導致由contextmanager提出了RuntimeException。至少finally可靠執行。

方法1：手動跳過正文。

@contextlib.contextmanager 
def drivercontext(): 
    driver, ok = driverfactory() 
    try: 
    yield driver, ok 
    finally: 
    driver.quit() 

with drivercontext() as (driver, ok): 
    if ok: 
    dostuff(driver) 
    else: 
    print 'skip because driver not ok' 

這雖然是顯式的，但它抵消了上下文管理器主體的大部分簡潔性。應該隱藏在上下文管理器中的邏輯泄露給主體，並且必須爲每個調用重複。方法2：濫用發電機。

def drivergenerator(): 
    driver, ok = driverfactory() 
    try: 
    if ok: 
     yield driver 
    else: 
     print 'skip because driver not ok' 
    finally: 
    driver.quit() 

for driver in drivergenerator(): 
    dostuff(driver) 

這種行爲很像一個上下文管理器，它可以跳過身體。不幸的是，它看起來非常像一個循環。方法3：手動做所有事情。

driver, ok = driverfactory() 
try: 
    if ok: 
    dostuff(driver) 
    else: 
    print 'skip because driver not ok' 
finally: 
    driver.quit() 

Bah。這是什麼？詳細程度可以與Java相媲美。

推廣這隻能用回調來完成。

def withdriver(callback): 
    driver, ok = driverfactory() 
    try: 
    if ok: 
     callback(driver) 
    else: 
     print 'skip because driver not ok' 
    finally: 
    driver.quit() 

withdriver(dostuff) 

唉。上下文管理器抽象了很多情況。但總是會有裂縫。這讓我想起了the law of leaky abstractions。

---

下面是一些代碼演示這些方法和一些其他方法。

import contextlib 
import functools 

# ---------------------------------------------------------------------- 
# this code is a simulation of the code not under my control 
# report and ok and fail are variables for use in the simulation 
# they do not exist in the real code 
# report is used to report certain checkpoints 
# ok is used to tell the driver object whether it is ok or not 
# fail is used tell dostuff whether it should fail or not 

class Driver(object): 
    def __init__(self, report, ok): 
    # driver can be ok or not ok 
    # driver must always quit after use 
    # regardless if it is ok or not 
    print 'driver init (ok: %s)' % ok 
    self.report = report 

    def drivestuff(self): 
    # if driver is not ok it is not ok to do stuff with it 
    self.report.drivestuffrun = True 

    def quit(self): 
    # driver must always quit regardless of ok or not 
    print 'driver quit' 
    self.report.driverquit = True 

def driverfactory(report, ok=True): 
    # driver factory always returns a driver 
    # but sometimes driver is not ok 
    # this is indicated by second return value 
    # not ok driver must still be quit 
    return Driver(report, ok), ok 

class DoStuffFail(Exception): 
    pass 

def dostuff(driver, fail=False): 
    # this method does a lot of stuff 
    # dostuff expects an ok driver 
    # it does not check whether the driver is ok 
    driver.drivestuff() 
    # do stuff can also fail independent of driver 
    if fail: 
    print 'dostuff fail' 
    raise DoStuffFail('doing stuff fail') 
    else: 
    print 'dostuff' 

# ---------------------------------------------------------------------- 
class AbstractScenario(object): 
    def __init__(self, driverfactory, dostuff): 
    self.driverfactory = functools.partial(driverfactory, report=self) 
    self.dostuff = dostuff 
    self.driverquit = False 
    self.drivestuffrun = False 

# ---------------------------------------------------------------------- 
class Scenario0(AbstractScenario): 

    def run(self): 
    print '>>>> not check driver ok and not ensure driver quit' 
    driver, ok = self.driverfactory() 
    self.dostuff(driver) 
    driver.quit() 

# ---------------------------------------------------------------------- 
class Scenario1(AbstractScenario): 

    def run(self): 
    print '>>>> check driver ok but not ensure driver quit' 
    driver, ok = self.driverfactory() 
    if ok: 
     self.dostuff(driver) 
    else: 
     print 'skip because driver not ok' 
    driver.quit() 

# ---------------------------------------------------------------------- 
class Scenario2(AbstractScenario): 

    def run(self): 
    print '>>>> check driver ok and ensure driver quit' 
    driver, ok = self.driverfactory() 
    try: 
     if ok: 
     self.dostuff(driver) 
     else: 
     print 'skip because driver not ok' 
    finally: 
     driver.quit() 

# ---------------------------------------------------------------------- 
class Scenario3(AbstractScenario): 

    @contextlib.contextmanager 
    def drivercontext(self, driverfactory): 
    driver, ok = driverfactory() 
    try: 
     if ok: 
     yield driver 
     else: 
     print 'skip because driver not ok' 
    finally: 
     driver.quit() 

    def run(self): 
    print '>>>> skip body by not yielding (does not work)' 
    with self.drivercontext(self.driverfactory) as driver: 
     self.dostuff(driver) 

# ---------------------------------------------------------------------- 
class Scenario4(AbstractScenario): 

    @contextlib.contextmanager 
    def drivercontext(self, driverfactory): 
    driver, ok = driverfactory() 
    try: 
     yield driver, ok 
    finally: 
     driver.quit() 

    def run(self): 
    print '>>>> skip body manually by returning flag with context' 
    with self.drivercontext(self.driverfactory) as (driver, ok): 
     if ok: 
     self.dostuff(driver) 
     else: 
     print 'skip because driver not ok' 

# ---------------------------------------------------------------------- 
class Scenario5(AbstractScenario): 

    def drivergenerator(self, driverfactory): 
    driver, ok = driverfactory() 
    try: 
     if ok: 
     yield driver 
     else: 
     print 'skip because driver not ok' 
    finally: 
     driver.quit() 

    def run(self): 
    print '>>>> abuse generator as context manager' 
    for driver in self.drivergenerator(self.driverfactory): 
     self.dostuff(driver) 

# ---------------------------------------------------------------------- 
def doscenarios(driverfactory, dostuff, drivestuffrunexpected=True): 
    for Scenario in AbstractScenario.__subclasses__(): 
    print '-----------------------------------' 
    scenario = Scenario(driverfactory, dostuff) 
    try: 
     try: 
     scenario.run() 
     except DoStuffFail as e: 
     print 'dostuff fail is ok' 
     if not scenario.driverquit: 
     print '---- fail: driver did not quit' 
     if not scenario.drivestuffrun and drivestuffrunexpected: 
     print '---- fail: drivestuff did not run' 
     if scenario.drivestuffrun and not drivestuffrunexpected: 
     print '---- fail: drivestuff did run' 
    except Exception as e: 
     print '----- fail with exception' 
     print '--------', e 

# ---------------------------------------------------------------------- 
notokdriverfactory = functools.partial(driverfactory, ok=False) 
dostufffail = functools.partial(dostuff, fail=True) 

print '============================================' 
print '==== driver ok and do stuff will not fail ==' 
doscenarios(driverfactory, dostuff) 

print '============================================' 
print '==== do stuff will fail =================' 
doscenarios(driverfactory, dostufffail) 

print '===========================================' 
print '===== driver is not ok ===================' 
doscenarios(notokdriverfactory, dostuff, drivestuffrunexpected=False) 

並輸出。

============================================ 
==== driver ok and do stuff will not fail == 
----------------------------------- 
>>>> not check driver ok and not ensure driver quit 
driver init (ok: True) 
dostuff 
driver quit 
----------------------------------- 
>>>> check driver ok but not ensure driver quit 
driver init (ok: True) 
dostuff 
driver quit 
----------------------------------- 
>>>> check driver ok and ensure driver quit 
driver init (ok: True) 
dostuff 
driver quit 
----------------------------------- 
>>>> skip body by not yielding (does not work) 
driver init (ok: True) 
dostuff 
driver quit 
----------------------------------- 
>>>> skip body manually by returning flag with context 
driver init (ok: True) 
dostuff 
driver quit 
----------------------------------- 
>>>> abuse generator as context manager 
driver init (ok: True) 
dostuff 
driver quit 
============================================ 
==== do stuff will fail ================= 
----------------------------------- 
>>>> not check driver ok and not ensure driver quit 
driver init (ok: True) 
dostuff fail 
dostuff fail is ok 
---- fail: driver did not quit 
----------------------------------- 
>>>> check driver ok but not ensure driver quit 
driver init (ok: True) 
dostuff fail 
dostuff fail is ok 
---- fail: driver did not quit 
----------------------------------- 
>>>> check driver ok and ensure driver quit 
driver init (ok: True) 
dostuff fail 
driver quit 
dostuff fail is ok 
----------------------------------- 
>>>> skip body by not yielding (does not work) 
driver init (ok: True) 
dostuff fail 
driver quit 
dostuff fail is ok 
----------------------------------- 
>>>> skip body manually by returning flag with context 
driver init (ok: True) 
dostuff fail 
driver quit 
dostuff fail is ok 
----------------------------------- 
>>>> abuse generator as context manager 
driver init (ok: True) 
dostuff fail 
driver quit 
dostuff fail is ok 
=========================================== 
===== driver is not ok =================== 
----------------------------------- 
>>>> not check driver ok and not ensure driver quit 
driver init (ok: False) 
dostuff 
driver quit 
---- fail: drivestuff did run 
----------------------------------- 
>>>> check driver ok but not ensure driver quit 
driver init (ok: False) 
skip because driver not ok 
driver quit 
----------------------------------- 
>>>> check driver ok and ensure driver quit 
driver init (ok: False) 
skip because driver not ok 
driver quit 
----------------------------------- 
>>>> skip body by not yielding (does not work) 
driver init (ok: False) 
skip because driver not ok 
driver quit 
----- fail with exception 
-------- generator didn't yield 
----------------------------------- 
>>>> skip body manually by returning flag with context 
driver init (ok: False) 
skip because driver not ok 
driver quit 
----------------------------------- 
>>>> abuse generator as context manager 
driver init (ok: False) 
skip because driver not ok 
driver quit 

Answer 2

此功能似乎已被rejected。 Python開發者通常更喜歡明確的變體：

if need_more_workers(): 
    newcomm = get_new_comm(comm) 
    # ... 

您還可以使用高階函數：

def filter_comm(comm, nworkitems, callback): 
    if foo: 
     callback(get_new_comm()) 

# ... 

some_local_var = 5 
def do_work_with_newcomm(newcomm): 
    # we can access the local scope here 

filter_comm(comm, nworkitems, do_work_with_newcomm) 

Answer 3

如何這樣的事情，而不是：

@filter_comm(comm, nworkitems) 
def _(newcomm): # Name is unimportant - we'll never reference this by name. 
    ... do work with communicator newcomm... 

您實現filter_comm裝飾這些結果決定是否執行它的包裹功能做任何工作，它應該與comm和nworkitems，然後根據圍繞或不圍繞，通過newcomm。

這不是很優雅的with，但我認爲這是一個有點更具可讀性，更接近你想要比其他提案的內容。你能說出比_其他內部功能的東西，如果你不喜歡這個名字，可是我用它去，因爲它是在Python中的正常名當語法需要一個你永遠不會真正使用的名稱。