如果我有 val incomingIds : List[Int] = ....
val existingIds : List[Int] = //this makes db calls and find existing records (only interested in returning ids)
現在,未來我想與existingIds以下列方式 說我有 val incomingIds
我想通過滑動窗口將部分函數傳遞給在DStream批處理中捕獲的所有RDD的聯合。可以說,我對建設離散到1分二批流超過10秒的窗口操作: val ssc = new StreamingContext(new SparkConf(), Seconds(1))
val stream = ssc.socketStream(...)
val window = stream.window(Seconds(