SparkSQL | 表生成函数_python sparksql lateral view explode split

lateral view与 explode函数按理说是不应该在数据库里存在的，因为他违背了第一范式(每个属性不可再分)。但是实际的场景，如一些大数据场景还是会存在将一些低频使用但又不能丢失的数据存成json，这种场景下就需要解析json，将里面的数组和多key值展开。

初始化一份数据

# 随意造的一份数据，毫无意义
data = [
    {
   
        "id": 1,
        "name": "XiaoHua",
        "age": 12,
        "interests": "game,read,tv",
        "interests_socre": {
   'game': 8, 'read': 7, 'tv': 8},
        "scores": {
   
             "scores": [{
   
                    "subject": "math",
                    "score": 80
                }, {
   
                    "subject": "language",
                    "score": 90
                }, {
   
                    "subject": "sports",
                    "score": 70
            }],
            "count": 3
        },
        "scores_str": '[{"subject": "math", "score": 80}, {"subject": "language", "score": 90}, {"subject": "sports", "score": 70}]'
    },

    {
   
        "id": 2,
        "name": "QiangQiang",
        "age": 13,
        "in
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35