如何減少我的文本文件大小？

Question

我有一個從我的數據庫獲取產品ID的PHP代碼（23436個獨特記錄）。

我獲取每個產品ID，並通過比較productID來檢查它是否已在feature_product表中設置。

如果在功能表中沒有找到與該ID相關的記錄，則可以通過將文本文件中的productID與feature_product中不存在的productID進行比較來再次檢查產品缺失功能的trial.txt文件表。

問題是，trial.txt文件中有593262行，它永遠需要在此文件中匹配productID。我用完了內存。我花了15個小時纔將所有數據從文件中刪除，並且也手動部分刪除了這些數據。有什麼辦法可以讓它更快或者沒有耗盡時間和內存？

我試圖增加我的php.ini文件中的最大執行時間，如網站上的一些帖子所建議的。但是它會持續耗盡內存或最大執行時間。一旦我得到這個權利，我將使用mysqli，因爲mysql不再被使用。我想劃分產品ID，所以我一次只能循環說5000，但我認爲這對執行時間沒有幫助。

例如產品表實施例樣品的

id_product,shop,manufacutrer,category 
1000010,1,41,1112,1 
1000011,1,7,1721,1 
1000012,1,7,1721,1 

特徵表

feature_id,id_product,value 
1,1000010,1 
3,1000010,2 
6,1000011,5 
11,1931555,1 

爲trial.txt的

<?php $conn = mysql_connect("localhost", "dbuser", "pwd"); //loop through the 1st line to avoid the headers in csv if (!$conn){ die('Could not connect : ' . mysql_error()); echo mysql_error(); } echo '<p>Connected!'; mysql_select_db("mydb") or die("Unable to select database"); //Select all product ids from product table into product array $pArray = mysql_query("SELECT `id_product` from `product`",$conn); //loop through each product id while($row = mysql_fetch_assoc($pArray)) { //get product ID to check if it exists in features table $productID = $row["id_product"]; //check whether product id exists in feature table where product_id matches both product table and features table $fArray = mysql_query("SELECT * from `feature_product` WHERE `id_product`=$productID"); //if product Id does not have entry in feature table than call a function to get check if product id has features in text file if(mysql_num_rows($fArray) ==0) { checkFeatures($productID); } else continue; } function checkFeatures($productID){ //trial.txt contains features of the products that are missing in features table but the products are in products table $fd = fopen('trial.txt', 'r'); $fheader = fgets($fd); //creates a new text file to save all features(multiple records per product) separated by ',' for future use $my_file = 'file.txt'; $handle = fopen($my_file, 'a') or die('Cannot open file: '.$my_file); while (($data = fgetcsv($fd,0, "~")) !== FALSE) { //Since this text file has many products i only get the ones that are missing in the features table by comparing product ID which is the 1st element of data array if($data[0]==$productID){ $d= $data[0].",".$data[1].",".$data[2].$data[3]."\n"; echo $d."<BR/>"; fwrite($handle, $d); } } fclose($fd); fclose($handle); } ?> 

IMSKU~AttributeID~Value~Unit~StoredValue~StoredUnit 
1000006~16121~2-25~~~ 
1000006~3897~* McAfee Protection Suite~~~ 
1000006~3933~* 1yr Subscription~~~ 
1000010~1708~Feb 2011~~~ 
1000010~1710~Cisco~~0.00~ 
1000010~1711~http://www.cisco.com~~~ 
1000011~2852~1~~0.00~ 
1000011~2855~Light Cyan~~0.00~ 
1000012~2840~May 2010~~~ 
1000012~2842~HP~~0.00~ 

我試圖加載文本文件在SQL表由用戶

<?php $con=mysqli_connect("localhost","username","pwd","db"); 
// Check connection 
if (mysqli_connect_errno()) 
{ 
echo "Failed: " . mysqli_connect_error(); 
} 

mysqli_query($con,"CREATE TABLE IF NOT EXISTS `add_features` (`id_product` INT(10) NOT NULL, `id_feature` INT(10) NOT NULL, `value` varchar(255),`unit` varchar(20),`s_value` varchar(20),`s_unit` varchar(20))"); 

$sql = "LOAD DATA INFILE 'trial.txt' 
INTO TABLE `add_features` 
FIELDS TERMINATED BY '~' 
"; 
if ($con->query($sql) === TRUE) { 
echo "OK!"; 
} else { 
echo "Error: " . $sql . "<br>" . $con->error; 
} 
$result = mysqli_query($con,"SELECT * FROM `add_features`"); 

echo "<table class='add_features'> 
<tr class='titles'> 
<th>Product_id</th> 
<th>feature_id</th> 
<th>value</th> 
<th>Unit</th> 
</tr>"; 

while($row = mysqli_fetch_array($result)) 
{ 
echo "<tr>"; 
echo "<td>" . $row['id_product'] . "</td>"; 
echo "<td>" . $row['id_feature'] . "</td>"; 
echo "<td>" . $row['value'] . "</td>"; 
echo "<td>" . $row['unit'] . "</td>"; 
echo "</tr>"; 
} 
echo "</table>"; 

mysqli_close($con); 
?> 

但我得到一個錯誤
錯誤的建議：LOAD DATA INFILE '爲trial.txt' INTO TABLE add_features FIELDS '〜'

Answer 1

如果trial.txt文件是靜態的，我會根據某個邏輯分隔符處理它/分解成單獨的較小文件，或者將它導入到一個新的數據庫表（最好），在其中搜索它將是即時的。這是一次導入，然後完成。

如果它不是靜態的，它多久改變一次？