如何應對這種日誌包含JSON

Question

這裏是日誌：

"V2|ip-10-203-5-16|PSDK_KINESIS_LOG|PSDK_DEFAULT|2016-05-12 00:00:05|VP|aa2ddfdb-4387-49c0-9651-92cc83b8e905|{"vid":"2237031","fdn":"FDNB1995115","type":"0","version":"1.0","device_id":"SCL-TL00_a0-8d-16-e6-39-13_868145026279574","ip":"58.254.0.157","timestamp":1462982395}" 

我的代碼：

import com.fasterxml.jackson.annotation.JsonProperty 
import com.fasterxml.jackson.core.JsonParseException 
import com.fasterxml.jackson.databind.ObjectMapper 
import org.apache.spark.{SparkContext, SparkConf} 
import org.slf4j.LoggerFactory 

class JsonLong{ 
    @JsonProperty var fdn: String = null 
    @JsonProperty("type") var typ: String = null 
    @JsonProperty var vid: String = null 
    @JsonProperty var version: String = null 
    @JsonProperty var device_id: String = null 
    @JsonProperty var ip: String = null 
    @JsonProperty var timestamp: Long = 0L 
    override def toString = s"JsonLong(fdn=$fdn, typ=$typ, vid=$vid,version=$version, device_id=$device_id, ip=$ip, timestamp=$timestamp)" 
} 



def jsonString(args:String):JsonLong ={ 
    val mapper = new ObjectMapper() 
    val record = mapper.readValue(args, classOf[JsonLong]) 
    record 
    } 

case class log(log_version: String,log_ip: String,log_from: String,SDK: String,action_time: Date,action: String,sn: String,post_code: JsonLong) 

val df = new SimpleDateFormat("yyyy-mm-dd HH:mm:ss") 
val input = sc.textFile("input.snappy") 
val RDD = input.map { line => 
    val p = line.split("\\|") 
    val log_version = p(0) 
    val log_ip = p(1) 
    val log_from = p(2) 
    val SDK = p(3) 
    val action_time = df.parse(p(4)) 
    val action = p(5) 
    val sn = p(6) 
    val post_code = if(p.length==8){ 
    //to read the last JSON 
    jsonString(p(7)) 
    } else("null") 
    log(log_version,log_ip,log_from,SDK,new Date(action_time.getTime()),action,sn,post_code)}.toDF() 

我有最後的Json問題。我已經確定jsonString返回calss JsonLong，但它返回Object。如何處理最後的Json？

Answer 1

用來初始化post_code不一定返回一個JsonLong表達：

if(p.length==8) { 
    jsonString(p(7)) // returns JsonLong 
} else("null")  // returns String 

所以，編譯器推斷，將適合兩個結果，在這種情況下java.lang.Object最好的類型。

我假設你打算使用null代替"null"，這將解決這個問題：

if(p.length==8) { 
    jsonString(p(7)) // returns JsonLong 
} else { null }  // returns Null, which extends all classes including JsonLong 

這將使編譯器推斷類型JsonLong爲post_code，這應該解決您的問題。