星火數據框中小數精度

Question

我有一個數據幀：

val groupby = df.groupBy($"column1",$"Date")  
.agg(sum("amount").as("amount")) 
.orderBy($"column1",desc("cob_date")) 

當applyin窗函數添加新列差：

val windowspec= Window.partitionBy("column1").orderBy(desc("DATE")) 

groupby.withColumn("diffrence" ,lead($"amount", 1,0).over(windowspec)).show() 


+--------+------------+-----------+--------------------------+ 
| Column | Date  | Amount | Difference    | 
+--------+------------+-----------+--------------------------+ 
| A  | 3/31/2017 | 12345.45 | 3456.540000000000000000 | 
+--------+------------+-----------+--------------------------+ 
| A  | 2/28/2017 | 3456.54 | 34289.430000000000000000 | 
+--------+------------+-----------+--------------------------+ 
| A  | 1/31/2017 | 34289.43 | 45673.987000000000000000 | 
+--------+------------+-----------+--------------------------+ 
| A  | 12/31/2016 | 45673.987 | 0.00E+00     | 
+--------+------------+-----------+--------------------------+ 

我越來越小數與尾部零。當我做printSchema()獲取數據類型的差異：decimal(38,18)。有人告訴我如何將數據類型更改爲decimal(38,2)或刪除尾隨零

Answer 1

你可以像下面的具體小數的大小投的數據，

鉛（$ 「量」，1,0）.over（windowspec）.cast（DataTypes.createDecimalType（32,2））

Answer 2

在純SQL中，可以使用公知技術：

SELECT ceil(100 * column_name_double)/100 AS cost ...