Quantcast
Channel: How can I change column types in Spark SQL's DataFrame? - Stack Overflow
Viewing all articles
Browse latest Browse all 24

Answer by cubic lettuce for How can I change column types in Spark SQL's DataFrame?

$
0
0

In case you have to rename dozens of columns given by their name, the following example takes the approach of @dnlbrky and applies it to several columns at once:

df.selectExpr(df.columns.map(cn => {    if (Set("speed", "weight", "height").contains(cn)) s"cast($cn as double) as $cn"    else if (Set("isActive", "hasDevice").contains(cn)) s"cast($cn as boolean) as $cn"    else cn}):_*)

Uncasted columns are kept unchanged. All columns stay in their original order.


Viewing all articles
Browse latest Browse all 24

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>