並行讀取多個文件並將它們存儲在共享變量中

Question

我有一個程序從另一個MPI程序生成的大量文件（〜100s文件，每個120MB）中讀取，這可能需要一些時間。每個文件都包含相應子域中的變量。我想讀取這些文件中的變量並將它們存儲到4維數組的特定片段中。由於需要一些相當長的時間，我想並行這段代碼與openmp：

6  SUBROUTINE read_old_restart 
    7   INTEGER :: ii 
    8   INTEGER :: thread_ID 
    9   INTEGER :: OMP_GET_THREAD_NUM 
10   CHARACTER(LEN=21) :: file_name 
11 
12   !$OMP PARALLEL DO PRIVATE(ii,file_name) 
13   DO ii=0,Nproc_old-1 
14    IF(ii < 10) THEN 
15     WRITE(file_name,401) "input/Restart_00", ii, ".out" 
16    ELSE IF(ii < 100) THEN 
17     WRITE(file_name,402) "input/Restart_0" , ii, ".out" 
18    ELSE 
19     WRITE(file_name,403) "input/Restart_" , ii, ".out" 
20    END IF 
21    PRINT*, "Thread = ", OMP_GET_THREAD_NUM(), "Reading ", file_name 
22 401 format(a16,I1,a4) 
23 402 format(a15,I2,a4) 
24 403 format(a14,I3,a4) 
25    OPEN (unit=321, file=TRIM(file_name), status="old", form="unFORMATted") 
26    READ(321) t      ,       & 
27      old_u   (:,:,:,ii),       & 
28      old_v   (:,:,:,ii),       & 
29      old_w   (:,:,:,ii),       & 
30      old_p   (:,:,:,ii),       & 
31      old_uc  (:,:,:,ii),       & 
32      old_vc  (:,:,:,ii),       & 
33      old_wc  (:,:,:,ii),       & 
34      old_un2  (:,:,:,ii),       & 
35      old_vn2  (:,:,:,ii),       & 
36      old_wn2  (:,:,:,ii),       & 
37      old_un1  (:,:,:,ii),       & 
38      old_vn1  (:,:,:,ii),       & 
39      old_wn1  (:,:,:,ii),       & 
40      old_p1  (:,:,:,ii),       & 
41      old_viscu  (:,:,:,ii),       & 
42      old_viscv  (:,:,:,ii),       & 
43      old_viscw  (:,:,:,ii),       & 
44      old_convu  (:,:,:,ii),       & 
45      old_convv  (:,:,:,ii),       & 
46      old_convw  (:,:,:,ii),       & 
47      statindex    ,       & 
48      old_umn  (:,:,:,ii),       & 
49      old_uumn  (:,:,:,ii),       & 
50      old_urms  (:,:,:,ii),       & 
51      old_mass_frac (:,:,:,:,:,ii), &     
52      old_enthT  (:,:,:,:,ii) 
53    CLOSE (321) 
54   END DO 
55   !$OMP END PARALLEL DO 
56  END SUBROUTINE read_old_restart 

代碼編譯和運行良好的每個線程上的第一圈。這裏是輸出：

Thread =   3 Reading input/Restart_030.out 
Thread =   7 Reading input/Restart_067.out 
Thread =   2 Reading input/Restart_020.out 
Thread =   6 Reading input/Restart_058.out 
Thread =   9 Reading input/Restart_085.out 
Thread =   8 Reading input/Restart_076.out 
Thread =   5 Reading input/Restart_049.out 
Thread =   4 Reading input/Restart_040.out 
Thread =   11 Reading input/Restart_103.out 
Thread =   0 Reading input/Restart_000.out 
Thread =   1 Reading input/Restart_010.out 
Thread =   10 Reading input/Restart_094.out 

該代碼似乎運行並卡住上述輸出。運行頂部時，我不能使用任何CPU。任何想法爲什麼它不按預期工作？

Answer 1

您應該使用私有整數變量作爲單位編號，並將其設置爲每個線程的不同值。使用與不同線程不同的文件單元是麻煩的祕訣。我很驚訝它不會崩潰。