如何使用一個或多個StructType創建模式（StructType）Spark

Question

我想在另一個StructType內創建StructType，但它只允許添加StructField。我找不到任何方法將StructType添加到它。

如何爲下面的字符串表示創建StructType模式？

struct<abc:struct<name:string>,pqr:struct<address:string>> 

Answer 1

有星火SQL的這個隱藏的功能來定義使用所謂架構DSL的模式（即沒有很多圓括號等）。

scala> spark.version 
res0: String = 2.2.0 

import org.apache.spark.sql.types._ 
val name = new StructType().add($"name".string) 
scala> println(name.simpleString) 
struct<name:string> 

val address = new StructType().add($"address".string) 
scala> println(address.simpleString) 
struct<address:string> 

val schema = new StructType().add("abc", name).add("pqr", address) 
scala> println(schema.simpleString) 
struct<abc:struct<name:string>,pqr:struct<address:string>> 

scala> schema.simpleString == "struct<abc:struct<name:string>,pqr:struct<address:string>>" 
res4: Boolean = true 

scala> schema.printTreeString 
root 
|-- abc: struct (nullable = true) 
| |-- name: string (nullable = true) 
|-- pqr: struct (nullable = true) 
| |-- address: string (nullable = true) 

Answer 2

structField是類型和名稱的組合，所以你會怎麼做：

StructType(Seq(StructField("structName", StructType(Seq(StructField("name", StringType), StructField("address", StringType))))