PySpark – How to Handle Non-Ascii Characters and connect in a Spark Dataframe?

Below code snippet tells you how to convert NonAscii characters to Regular String and develop a table using Spark Data frame. I have created a small udf and register it in pyspark. Please see the code below and output.

screen-shot-2016-09-15-at-12-25-25-pm

 

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s