記號化/ untokenize Python字符串代碼，因此它與交互模式

Question

兼容我有這樣的Python代碼具有這種結構的

def main: 

    ''' comment ''' 
    if True: 
     print "do" 
    print "done 

此代碼是不是與交互模式（例如，如果兼容我將其複製/粘貼到交互式會話中）。爲此，它需要：

def main: 
    ''' comment ''' 
    if True: 
     print "do" 

    print "done" 

否則交互式模式會破壞縮進問題。

你知道一個簡單的方法來轉換代碼與generate_token/untokenize鏈嗎？我有點迷失在NL/NEWLINE/INDENT/DEDENT語義。

我發現這Script to remove Python comments/docstrings刪除評論/文檔字符串。它看起來非常適合我的問題，但它無法對複雜代碼進行清理輸出。

Answer 1

我能拿出最好的（解決了我的問題）

def _python_interactive_indent(self, code): 
    prev_toktype = tokenize.INDENT 
    first_line = None 
    last_lineno = -1 
    last_col = 0 

    output = '' 

    tokgen = tokenize.generate_tokens(StringIO.StringIO(code).readline) 
    indent = 0 
    hasNL = False 
    prefixed = False 
    for toktype, ttext, (slineno, scol), (elineno, ecol), ltext in tokgen: 
     done = False 
     if toktype == tokenize.INDENT: 
      indent = indent + 1 
     if toktype == tokenize.DEDENT: 
      indent = indent - 1 
     if slineno > last_lineno: 
      last_col = 0 
     if not done and toktype == tokenize.NL: 
      hasNL = True 
      done = True 
     if not done and toktype == tokenize.COMMENT: 
      done = True 
     if not done and toktype == tokenize.STRING and prev_toktype == tokenize.INDENT: 
      done = True 
     if not done and hasNL and toktype != tokenize.DEDENT and toktype != tokenize.INDENT: 
      hasNL = False 
      output = output + (" " * indent) + '\n' 
      output += " " * indent 
      prefixed = True 
     if not done: 
      if not prefixed and scol > last_col: 
       output += (" " * (scol - last_col)) 
      output += (ttext) 
     prefixed = False 
     prev_toktype = toktype 
     last_col = ecol 
     last_lineno = elineno 
    return output