Android平行（同步）圖像下載

Question

夥計們幫忙請它很重要。

我需要在不到5分鐘的時間內下載約2000張圖片。所以我決定下載並行圖像。

如果有人可以建議我更好地進行並行下載或告訴我我要出錯的地方，我真的很感激。

在完成的任務的錯誤日誌數總是各種各樣的...

請幫助。

我使用異步任務執行由阿爾喬姆Zinnatullin

public class AsyncTaskExecutor { 

private static final int CORE_POOL_SIZE; 
private static final int MAXIMUM_POOL_SIZE; 
private static final int KEEP_ALIVE; 
private static final TimeUnit TIME_UNIT; 

private static final BlockingQueue<Runnable> concurrentPoolWorkQueue; 
private static final ThreadFactory concurrentThreadFactory; 
    private static final ThreadPoolExecutor concurrentExecutor; 

private AsyncTaskExecutor() {} 

static { 
CORE_POOL_SIZE = 5; 
MAXIMUM_POOL_SIZE = 128; 
KEEP_ALIVE = 1; 
TIME_UNIT = TimeUnit.SECONDS; 

concurrentPoolWorkQueue = new LinkedBlockingQueue<Runnable>(10); 
concurrentThreadFactory = new AsyncTaskThreadFactory(); 
concurrentExecutor = new ThreadPoolExecutor(CORE_POOL_SIZE,MAXIMUM_POOL_SIZE, KEEP_ALIVE, TIME_UNIT, concurrentPoolWorkQueue, concurrentThreadFactory); 
} 
/** 
* Concurrently executes AsyncTask on any Android version 
* @param task to execute 
* @param params for task 
* @return executing AsyncTask 
*/ 
@SuppressLint("NewApi") 
public static <Params, Progress, Result> AsyncTask<Params, Progress, Result> 
executeConcurrently(AsyncTask<Params, Progress, Result> task, 
Params... params) { 
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.HONEYCOMB) { 
task.executeOnExecutor(concurrentExecutor, params); 
} else { 
task.execute(params); 
} 
return task; 
} 

    /** 
* Thread factory for AsyncTaskExecutor 
* @author Artem Zinnatullin 
* 
*/ 
private static class AsyncTaskThreadFactory implements ThreadFactory { 
private final AtomicInteger count; 

{ 
count = new AtomicInteger(1); 
} 

@Override 
public Thread newThread(Runnable r) { 
return new Thread(r, "AsyncTask #" + count.getAndIncrement()); 
} 
} 

}

然後我使用循環下載我的所有圖像。

for (PhotoToDBHelper photoToDBHelper : notDownloaded) { 
      try { 
       photoToDBHelper.download(m_context); 
      } catch (Exception e) { 
       // TODO: handle exception 
      } 

     } 

哪裏下載方法是這樣的：

public void download(Context context) throws Exception { 
    HttpClient httpClient = new DefaultHttpClient(); 
    HttpGet httpGet = new HttpGet(new URL(getPhotosUrl() + m_fileName).toString()); 
    HttpResponse response = httpClient.execute(httpGet); 
    HttpEntity entity = response.getEntity(); 

    InputStream inputStream = entity.getContent(); 
    filePath = Environment.getExternalStorageDirectory().toString() + getFoldeRelativePath() + m_fileName; 

    Utils.createFileOnExternalStorage(getFoldeRelativePath(), filePath); 
    FileOutputStream fos = new FileOutputStream(filePath, false); 

    byte[] buffer = new byte[20000]; 
    int byteRead = 0; 

    while ((byteRead = inputStream.read(buffer)) != -1) { 
     fos.write(buffer, 0, byteRead); 
    } 
    fos.close(); 
} 

當開始我的Android應用程序，我得到這個錯誤

02-25 01:08:30.133: E/AndroidRuntime(32357): FATAL EXCEPTION: main 
02-25 01:08:30.133: E/AndroidRuntime(32357): java.lang.RuntimeException: Unable to start activity ComponentInfo{"package".TabHostActivity}: java.util.concurrent.RejectedExecutionException: Task android.os.AsyncTask$3@2c0809b0 rejected from java.util.concurrent.ThreadPoolExecutor@2be18f48[Running, pool size = 128, active threads = 1, queued tasks = 10, completed tasks = 27] 
02-25 01:08:30.133: E/AndroidRuntime(32357): at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:1967) 
02-25 01:08:30.133: E/AndroidRuntime(32357): at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:1992) 
02-25 01:08:30.133: E/AndroidRuntime(32357): at android.app.ActivityThread.access$600(ActivityThread.java:127) 
02-25 01:08:30.133: E/AndroidRuntime(32357): at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1158) 
02-25 01:08:30.133: E/AndroidRuntime(32357): at android.os.Handler.dispatchMessage(Handler.java:99) 
02-25 01:08:30.133: E/AndroidRuntime(32357): at android.os.Looper.loop(Looper.java:137) 
02-25 01:08:30.133: E/AndroidRuntime(32357): at android.app.ActivityThread.main(ActivityThread.java:4441) 
02-25 01:08:30.133: E/AndroidRuntime(32357): at java.lang.reflect.Method.invokeNative(Native Method) 
02-25 01:08:30.133: E/AndroidRuntime(32357): at java.lang.reflect.Method.invoke(Method.java:511) 
02-25 01:08:30.133: E/AndroidRuntime(32357): at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:784) 
02-25 01:08:30.133: E/AndroidRuntime(32357): at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:551) 
02-25 01:08:30.133: E/AndroidRuntime(32357): at dalvik.system.NativeStart.main(Native Method) 
02-25 01:08:30.133: E/AndroidRuntime(32357): Caused by: java.util.concurrent.RejectedExecutionException: Task android.os.AsyncTask$3@2c0809b0 rejected from java.util.concurrent.ThreadPoolExecutor@2be18f48[Running, pool size = 128, active threads = 1, queued tasks = 10, completed tasks = 1032] 
02-25 01:08:30.133: E/AndroidRuntime(32357): at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:1967) 
02-25 01:08:30.133: E/AndroidRuntime(32357): at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:782) 
02-25 01:08:30.133: E/AndroidRuntime(32357): at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1303) 
02-25 01:08:30.133: E/AndroidRuntime(32357): at android.os.AsyncTask.executeOnExecutor(AsyncTask.java:564) 
02-25 01:08:30.133: E/AndroidRuntime(32357): at "package".data.AsyncTaskExecutor.executeConcurrently(AsyncTaskExecutor.java:58) 
02-25 01:08:30.133: E/AndroidRuntime(32357): at "package".TabHostActivity.checkUpdateState(TabHostActivity.java:130) 
02-25 01:08:30.133: E/AndroidRuntime(32357): at "package".TabHostActivity.onCreate(TabHostActivity.java:88) 
02-25 01:08:30.133: E/AndroidRuntime(32357): at android.app.Activity.performCreate(Activity.java:4465) 
02-25 01:08:30.133: E/AndroidRuntime(32357): at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1049) 
02-25 01:08:30.133: E/AndroidRuntime(32357): at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:1931) 
02-25 01:08:30.133: E/AndroidRuntime(32357): ... 11 more 

Answer 1

有一些需要解決的幾個問題：

1. 2000張圖片在5張分鐘約爲每秒7張圖像。根據圖像大小，網絡帶寬，處理器和存儲速度等因素，這可能不適用於任何代碼或併發量。吞吐量的數學公式爲：每秒7 * N字節，其中N是平均圖像大小。如果有大的圖像，它可能是一個死角。
2. 該異常表示線程池已達到最大排隊任務數（10）。這可以基於線程池管理器實現來配置。總的來說，試圖同時啓動2000個線程可能會加劇這種情況。建議隊列中的排隊任務（使用線程），監控它們的完成情況，然後開始更多，一次一個小組地說五（5）個小組。
3. 如果有任何服務器控制，請考慮使用精靈模型，其中多個圖像存儲在一個圖像（文件/資源​​）中，由圖像內的相對x/y座標分開。這將使HTTP或任何網絡請求的數量從2000減少到1（可能），從而顯着提高性能。