Quantcast
Channel: How can I change column types in Spark SQL's DataFrame? - Stack Overflow
Viewing all articles
Browse latest Browse all 24

Answer by Aravind Krishnakumar for How can I change column types in Spark SQL's DataFrame?

$
0
0
    val fact_df = df.select($"data"(30) as "TopicTypeId", $"data"(31) as "TopicId",$"data"(21).cast(FloatType).as( "Data_Value_Std_Err")).rdd    //Schema to be applied to the table    val fact_schema = (new StructType).add("TopicTypeId", StringType).add("TopicId", StringType).add("Data_Value_Std_Err", FloatType)    val fact_table = sqlContext.createDataFrame(fact_df, fact_schema).dropDuplicates()

Viewing all articles
Browse latest Browse all 24

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>