2012-08-13 81 views
1

這是我的第一篇文章,所以請讓我知道是否有任何我應該添加到帖子,如果需要的話。Java *多線程與MySQL

我有一個程序從我的MySQL數據庫中抓取URL,並將它們發送到將圖像下載到桌面的類。

我有80,000張圖片需要獲取,而且我也希望利用多線程使其更快。

我已經到了某個地方,但我現在被卡住了,花了數小時的研究,因爲我是多線程的新品。

問題是程序運行時,它會經過循環處理前5個條目一遍又一遍。這恰好是ExecutorFixedPoolSize。

有人可以幫助我。

我的代碼如下。

public static void main(String[] args) { 

     try{ 

      countAllAltPicsNotSaved(); 
      System.out.println("Not saved: " + totalAltPicsNotSaved); 

      Class.forName("com.mysql.jdbc.Driver"); 
      Connection conn =DriverManager.getConnection(dbUrl + dbName, userName, password); 
      Statement s = conn.createStatement(); 
      s.executeQuery ("SELECT DISTINCT id from productsaltimages WHERE saved != 1"); 
      ResultSet rs = s.getResultSet(); 
      int saveCounter = 0, altImageCount = 0; 

      List<Thread> threads = new ArrayList<Thread>(); 



      while (rs.next()) { //Get ID 
       altImageCount = 0; //Product Alt Image Counter 
       String id = rs.getString("id"); //Product Table ID 

       Statement news = conn.createStatement(); //New Conn for get Alt Images From ID 
       news.executeQuery ("SELECT url from productsaltimages WHERE id ='"+id+"' AND saved != 5"); //Gets query from mySQL 

       ResultSet newrs = news.getResultSet(); //Resultset for AltImges for ID 

       ExecutorService executor = Executors.newFixedThreadPool(5); 
       Runnable task = null; 

       while (newrs.next()){ //Get Images 
        altImageCount++; //Increment ID Alt Image Counter 
        String url = newrs.getString("url"); 
        task = new DownloadImage(url, id, altImageCount); 
        executor.execute(task); 

       } 
      } 

     } catch (Exception e){ 
     } 

    } 

    public static void countAllAltPicsNotSaved() throws Exception { 
     try{ 
      totalAltPicsNotSaved=0; 
      Class.forName("com.mysql.jdbc.Driver"); 
      Connection conn =DriverManager.getConnection(dbUrl + dbName, userName, password); 
      Statement s = conn.createStatement(); 
      boolean sqlExecuteok = false; 
      s.executeQuery ("SELECT * from productsaltimages WHERE saved = '0'"); 
      ResultSet rs = s.getResultSet(); 
      while (rs.next()) { 
       totalAltPicsNotSaved++; 
      } 
      rs.close(); 
      s.close(); 
      conn.close(); 
     } catch (Exception e){ 
     } 
    } 


For my DownloadImage class code is: 

    public class DownloadImage implements Runnable 
{ 
    String url, id, threadName; 
    private int altImageCount = 0; 
    private static String userName = "owner"; 
    private static String password = "hello123"; 
    private static String dbUrl = "jdbc:mysql://localhost/"; 
    private static String dbName = "shop"; 
    private static String dbClass = "com.mysql.jdbc.Driver"; 

    public DownloadImage(String url, String id, int altImageCount) 
    { 
     this.url = url; 
     this.id = id; 
     this.altImageCount = altImageCount; 
    } 
    public void run() 
    { 
     while(true) 
     { 

     File file=new File("D:\\FBShop_AltImages\\" + id + "\\"); 
     boolean exists = file.exists(); 
     if (!exists) { 
      // Create multiple directories 
      boolean success = (new File("D:\\FBShop_AltImages\\" + id + "\\")).mkdirs(); 
     } 

     String newFilename = "D:\\FBShop_AltImages\\" + id + "\\" + id + "_" + altImageCount + ".jpg"; 

     try { 

      try { 
       BufferedImage image = null; 
       URL imgurl = new URL(url); 
       URLConnection con = imgurl.openConnection(); 
       con.setConnectTimeout(50 * 10000); 
       con.setReadTimeout(50 * 10000); 
       InputStream in = con.getInputStream(); 
       image = ImageIO.read(in); 
       ImageIO.write(image, "jpg", new File(newFilename)); 

      } catch (Exception e) { 
       System.out.println("Error getting image: " + url); 
      } 

      try { 
       //UpdateTable 
       Class.forName("com.mysql.jdbc.Driver"); 
       String updatesql = "UPDATE productsaltimages SET saved = '1' WHERE id = '"+id+"'"; 

       Connection conn =DriverManager.getConnection(dbUrl + dbName, userName, password); 
       Statement ups = conn.createStatement(); 
       int val = ups.executeUpdate(updatesql); 
       System.out.println("Task Complete: " + url); 
       try { 
        Thread.sleep(5000); 
       } catch (Exception e) { 
       } 
      } catch (Exception e) { 
      } 
     } catch (Exception e) { 
     } 
     //System.out.println("Thread Finished: " + threadName); 
     } 
    } 
} 
+0

@Eugene Lol,微妙的提示:p – Luc 2014-08-17 15:07:15

回答

2
while(true){ 
    //lots of code inside you Runnable 

    Thread.sleep(5000); 
    //After the Thread will sleep it will restart it's work again, and again.. 
} 

什麼時候會結束?除去一切正確,你應該很好。

+0

對不起,我沒有發生什麼,爲什麼那裏..因爲我學習我複製粘貼網絡的一些代碼..我想我在那裏學到了我的教訓不要做那!!我刪除了這一切,並一切順利..花了幾個小時,但至少我研究了多線程,並已學會了很多哈哈..謝謝你這是固定它對我來說。 – 2012-08-13 17:38:46

+0

@JeffreyHolmes如果你想了解Threading,你應該閱讀一本書 - Concurrency In Practice - 它是最好的。乾杯! – Eugene 2012-08-13 18:21:55

+1

酷男謝謝:D我可能會那樣做!順便說一句,我喜歡這個網站。 – 2012-08-13 18:38:11