2014-01-13 50 views
0

我和周圍10萬記錄的表,其結構如下所示爲PHP和MySQL優化內存管理

id | name | desc | length | breadth | -------------|remark //upt o 56 fields 
1  FRT-100 -desc-  10  10      remarking 
---------------------------------------------------------------- 
---------------------------------------------------------------- 

什麼IAM這樣做是使用一個cronjob(Gearman的)寫所有這些數據爲csv ,我的代碼如下

 <?php 
     set_time_limit (0); 
    ini_set("memory_limit","2048M"); 
     //get the total count of records,so that we can loop it in small chunks 
     $query = "SELECT COUNT(*) AS cnt FROM tablename WHERE company_id = $companyid"; 

      $result = $link->query($query); 
      $count = 0; 
      while ($row = mysqli_fetch_array($result)) { 
       $count = $row["cnt"]; 
      } 
      if ($count > 1000) { 
       $loop = ceil($count/1000); 
      } else { 
       $loop = 1; 
      } 


     // im going to write it in small chunks of 1000's each time to avoid time out 

      for ($ii = 1; $ii <= $loop; $ii++) { 

       if ($ii == 1) { 
        $s = 1; 
       } else { 
        $s = floatval(($ii * 1000) - 1000); 
       } 

       $q = "SELECT * FROM datas WHERE group_company_id = $companyid LIMIT 1000 OFFSET $s"; 
       $r = $link->query($q);  

     while ($row2 = mysqli_fetch_array($r)) { 
        //my csv writing will be done here and its working fine for records up to 10,000 ~ 12,000 after than memory exhaustion occours 
     } 
} 
    ?> 

給我強烈懷疑的東西可以在MySQL。可有人的偏移功能優化告訴我一個更好的方式來優化它?打開任何建議(CRON,第三方庫..等等)

+0

'while($ row = mysqli_fetch_array($ result)){'---爲什麼你需要一個循環在這裏? – zerkms

+0

@zerkms得到多少記錄的數量..因爲它的單行..你是對的 – coolguy

+0

@zerkms你可以把一些光的極限,偏移問題 – coolguy

回答

2

嘗試並避免將所有內容一次性存儲在內存中,而是加載每一行,然後一次寫出一行結果。

<?php 
$q = "SELECT * FROM datas"; 
$r = $link->query($q);  
$fp = fopen("out.csv","w+"); 
// Or you could just set the headers for content type, and echo the output 
while ($row2 = mysqli_fetch_array($r)) { 
    fwrite($fp, implode(",",$row2)."\n"); 
} 
fclose($fp); 

這應該解決這個問題,沒有什麼會被存儲在內存中。

+0

感謝生病嘗試 – coolguy