2017-09-23 28 views
1

所以目前我試着去得到一個功能權限。基本上,我嘗試做多(兼)線程需要的信息從一個JSON文件到程序,然後爲每個線程應該使用每個JSON對象,然後用這些信息來執行代碼。Python多線程/ Json和Python的流程組合?

我迄今所做的 - 這個代碼僅適用於多進程這的確作品:

#Read json File 
with open('profileMulti.json', 'r', encoding='UTF-8') as json_data: 
    profiles_string = json.load(json_data) 

def get_individual_profiles(config): 
    top_layer = config.get('Profiles') 
    if top_layer: 
     top_level_keys = ['profile_{}'.format(i) for i in range(len(top_layer))] 
     print(top_level_keys) 
     return [(key, top_layer.get(key)) for key in top_level_keys] 
    return [] 

def stringify(key, next_layer): 
    return [ 
     ' '.join(key.capitalize().split('_')), 
     next_layer.get('Name'), 
     next_layer.get('Last_Name'), 
     next_layer.get('Email'), 
     next_layer.get('Phone') 
     #etc etc... 
     ] 

    config = profiles_string 
profiles = get_individual_profiles(config) 

pool = ThreadPool() 

# Launch a process for each item 
threads = [pool.apply_async(stringify, tuple(item)) for item in profiles] 


# get() the results as each finishes 
results = [res.get() for res in threads] 
print('threaded results:') 
for item in results: 
    print(item) 

輸出:

threaded results: 
['Profile 0', 'Thrill', 'Ofit', '[email protected]', '123 412 123'] 

['Profile 1', 'Hellow', 'World', '[email protected]', '543 412 312'] 

這是偉大的。但問題是:

,每當我想等使用這些信息

爲例

def checkoutNames(NameUrl, nameID): 
payload = { 
     "shared": { 
      "challenge": { 
       "Name": item["Name"], 
       "Last_Name": item["Last_Name"], 
       "Email": item["Email"], 
       "Phone": item["Phone"], 

是它不會承認這些屬性,但我需要做的是,我需要由項目1 ..項目[2]等從以便從def stringify(key, next_layer):我不知道,現在叫他們怎麼做,所以我並不需要做。

的另一個問題是,每當我們說的ETC,我在代碼中使用項目1。然後它將只使用最後一個線程並跳過其餘部分。所以,如果我做

print(item[1]) 

那麼這會是唯一的輸出是最後一個是你好

所以我需要的問題得到解決:

1.每一線程併發執行,並與信息

2.執行代碼做一個修復,所以我並不需要使用項目1,而是使用項目['Name']。

所以現在的問題是,這是可能的,什麼是這個想法?

編輯 - 這是我目前只有一個Json的輪廓具有和工作正常只用一個配置文件的代碼。這是一個沒有多處理

JSON文件

 { 
    "Profiles": { 
     "profile_0": { 
      "Url": "Myownwebsite.se", 
      "My-Note": "Helloworld", 
      "Email": "[email protected]" 
      "PersonNumber": "1234543", 
      "postal_code": "54123", 
      "given_name": "World", 
      "Last_name": "Hellow", 
      "street_address": "helloworld 123", 
      "city": "Stockholm", 
      "country": "Sweden", 
      "phone": "123456789", 
      "Color": "Red", 
      "house_number": "123", 
      "year": "2017" 
     }, 
     "profile_1": { 
      "Url": "Myasdwfaesite.se", 
      "My-Note": "aasfase", 
      "Email": "[email protected]" 
      "PersonNumber": "5634543", 
      "postal_code": "123445", 
      "given_name": "Balling", 
      "Last_name": "Calling", 
      "street_address": "qwertr 123", 
      "city": "London", 
      "country": "UK", 
      "phone": "65412331", 
      "Color": "Blue", 
      "house_number": "321", 
      "year": "2018" 
     } 

     #Profile_2 etc etc 
    } 
} 

代碼

with open('profileMulti.json', 'r', encoding='UTF-8') as json_data: 
    config = json.load(json_data) 

NameUrl = config["Url"] 

myNote = config["My-Note"] 

def checkoutNames(NameUrl, nameID): 

#Request & other codes - Removed to recude the code 
#...... 
#...... 
    headers = { 
     'Referer': '', 
     'Content-Type': '' 
    } 
    payload = { 
     "shared": { 
      "challenge": { 
       "email": config["Email"], 
       "PersonNumber": config["PersonNumber"], 
       "postal_code": config["ZipCode"], 
       "given_name": config["Name"], 
       "Last_name": config["LastName"], 
       "street_address": config["Address"], 
       "postal_code": config["ZipCode"], 
       "city": config["City"], 
       "country": config["Country"], 
       "email": config["Email"], 
       "phone": config["Phone"], 
      } 

def checkoutNotes(NamesUrl, NamesPost): 

#Request & other codes - Removed to recude the code 
#...... 
#...... 

    headers = { 
     'Accept': 'application/json, text/javascript, /; q=0.01', 
     'Accept-Language': 'en-US,en;q=0.5', 
     'Accept-Encoding': 'gzip, deflate, br', 
     'Referer': NameUrl, 
     'Connection': 'keep-alive' 
    } 
    payloadInfo = { 
     "Information": { 
      "Color": config["Color"], 
      "house_number": config["houseNumber"], 
      "year": config["Year"] 
     } 
    }  
def wipe(): 
    os.system('cls' if os.name == 'nt' else 'clear') 

def main(): 
    time.sleep(1) 

    FindName(myNote) 

if _name_ == '_main_': 
    try: { 
     main() 
    } 
    except KeyboardInterrupt: 
     wipe() 

編輯

我只是想打印出什麼雅羅斯拉夫代碼提供輸出

enter image description here

回答

1

你做出列表並且像字典一樣工作。如果訂單很重要 - 從集合中使用。順便說一句,你可以加載你的JSON使用這樣的事情:

from collections import OrderedDict 
.... # your previous code 
profiles_string = json.load(json_data, object_pairs_hook=OrderedDict) 

嘗試使用此代碼:

import json 
from collections import OrderedDict 
from multiprocessing.pool import ThreadPool 
import threading 
from time import sleep 
import random 

with open('profileMulti.json', 'r', encoding='UTF-8') as json_data: 
    config = json.loads(json_data, object_pairs_hook=OrderedDict) 


def stringify(key, next_layer): 
    sleep(random.random()) # work emulation 
    # `key` is profile name (profile_0, profile_1 etc) 
    # `next_layer` is profile payload as OrderedDict: 
    # { 
    # "Url": "Myasdwfaesite.se", 
    # "My-Note": "aasfase", 
    # "Email": "[email protected]", 
    # "PersonNumber": "5634543", 
    # "postal_code": "123445", 
    # "given_name": "Balling", 
    # "Last_name": "Calling", 
    # "street_address": "qwertr 123", 
    # "city": "London", 
    # "country": "UK", 
    # "phone": "65412331", 
    # "Color": "Blue", 
    # "house_number": "321", 
    # "year": "2018" 
    #} 

    # this print statement just for illustartion 
    print(threading.current_thread().name, key) 
    return { key: next_layer } # still ordered 

# how much workers should be started 
profiles_number = len(config.get('Profiles', {}).items()) 

pool = ThreadPool(profiles_number) 

# Launch a thread for each item and get() the results as each finishes 
# `starmap_async` returns result object when all tasks are finished, so 
# single get is called. Result object in thios case behave like list 
results = pool.starmap_async(stringify, config.get('Profiles', {}).items()).get() 

print('threaded results:') 
for i in results: 
    print(i, end='\n***\n') # result separator when printed 
+0

謝謝@Yaroslav!但是,當我嘗試這一點。它不起作用。我得到的印刷品是[無,無]。什麼可能導致這個問題?也許是因爲'''print(item)'''沒有任何東西?也如果我想從線程打印出來的Url。在這種情況下不會打印(配置['Url'])完成它? – WeInThis

+0

@WeInThis,對不起,雙'返回'聲明是錯誤的。嘗試編輯的代碼,應該沒問題 –

+0

再次嘿!好吧,它的工作。但是我發現另一個問題。所以現在發生的事情是,它只是加載到一個線程中,並且在該線程內部是包括所有配置文件在內的所有對象的整個打印。我不認爲它應該這樣做。但也是。如果我想要抓取URL,情況會怎樣。等'''Url =「從profile_0抓取url」並打印(Url)'''? – WeInThis