SOLVED – Remove characters in from values pyspark

ByAman

Problem – How to remove characters from column values pyspark sql? Only values should come and values like 10-25 should come as it is 546,654,10-25

I.e gffg546, gfg6544

azure-synapse-analytics
azure-databricks
azure-data-lake-storage

You May Also Like: SOLVED – How To Get All Data From Last 5 Minutes in SQL

SOLUTION

You can use this with Spark Tables + Pandas DataFrames:

Example:

Documentation for converting spark to pd

https://docs.databricks.com/spark/latest/spark-sql/spark-pandas.html

You May Also Like: SOLVED – How To Update Connection Policy Of Synapse Dedicated SQL Pools

We hope the solutions in the above-mentioned article were helpful. Feel free to share your views with us.

Similar Posts

SOLVED – You have to delete all the items in this folder before you can delete the folder.

ByAman

For a folder on the team site., trying to delete it might show an error that says – “You have to delete all the items in this folder before you can delete the folder. Documents->Test1->Test2“. While trying to delete ‘Test2’ folder, the ‘Test2’ folder is empty and doesn’t have any files or folders in it….

SOLVED – How linq order by a Hyphen

ByAman

A list contains a structure like that: ColumnA 0+ABC0-ABC0001000200030004 then sort the list using order by like list. OrdeBy(o=>o.ColumnA).ToList(), the below is the result: 0+ABC00010002000300040-ABC So the question is why “0+ABC” at the top of the list but “0-ABC” is put to the end. it looks like linq sort “+” and “–” using different methods….

SOLVED – Using ubuntu-hpc images with Standard_ND40rs_v2 VMs

ByAman

This Ndv2 page[1] mentions that the 4.7-1.0.0.1 Mellanox OFED driver is what should be used when enabling infiniband on Standard_ND40rs_v2 VMs. However the ubuntu-hpc 18.04 images come pre-installed with v4.9-3.1.5.0 [2], and 20.04 images pre-installed come with v5.6-1.0.3.3 [3]. Problem – Is there a way to safely downgrade the Mellanox OFED driver without corrupting (or…

Leave a Reply

Your email address will not be published. Required fields are marked *