python多条件筛选数据filter_具有多个条件的python-sparksql筛选（使用where子句选择）...

作者：小蓝xlanll | 2024-04-03 12:28:39

踩

python dataframe 实现sql where 过滤

numeric = sqlContext.createDataFrame([

('3.5,', '5.0', 'null'), ('2.0', '14.0', 'null'), ('null', '38.0', 'null'),

('null', 'null', 'null'), ('1.0', 'null', '4.0')],

('low', 'high', 'normal'))

numeric_filtered_1 = numeric.where(numeric['LOW'] != 'null')

numeric_filtered_1.show()

+----+----+------+

| low|high|normal|

+----+----+------+

|3.5,| 5.0| null|

| 2.0|14.0| null|

| 1.0|null| 4.0|

+----+----+------+

numeric_filtered_2 = numeric_filtered_1.where(

numeric_filtered_1['NORMAL'] != 'null')

numeric_filtered_2.show()

+---+----+------+

|low|high|normal|

+---+----+------+

|1.0|null| 4.0|

+---+----+------+

numeric_filtered_3 = numeric_filtered_2.where(

numeric_filtered_2['HIGH'] != 'null')

numeric_filtered_3.show()

+---+----+------+

|low|high|normal|

+---+----+------+

声明：本文内容由网友自发贡献，不代表【wpsshop博客】立场，版权归原作者所有，本站不承担相应法律责任。如您发现有侵权的内容，请联系我们。转载请注明出处：https://www.wpsshop.cn/w/小蓝xlanll/article/detail/355534?site=