devbox
Monodyee
这个屌丝很懒，什么也没留下！
关注作者
热门标签
热门文章
当前位置:   article > 正文

树莓派4BUbuntu server 20.04 Kubernetes-v1.17.x- Docker19.03 keadmv1.10.3部署错误一览_your kernel does not support cgroup blkio weight_d

作者：Monodyee | 2024-03-28 11:33:50

赞

踩

your kernel does not support cgroup blkio weight_device

树莓派4BUbuntu server 20.04  keadmv1.10.3加入主节点报错

  1. execute keadm command failed:  failed to exec 'bash -c sudo ln /etc/kubeedge/edgecore.service /etc/systemd/system/edgecore.service && sudo systemctl daemon-reload && sudo systemctl enable edgecore && sudo systemctl start edgecore', err: ln: failed to create hard link '/etc/systemd/system/edgecore.service': File exists
  2. , err: exit status 1

解决办法删除'/etc/systemd/system/edgecore.service'重新加入节点，成功。

之后journalctl -u edgecore.service -xe 打印日志

报错

  1.  Version: v1.10.3
  2. Oct 10 16:09:23 node1 edgecore[241646]: I1010 16:09:23.205200  241646 server.go:103] Use node IP address from config: 192.168.137.74
  3. Oct 10 16:09:23 node1 edgecore[241646]: I1010 16:09:23.205291  241646 sql.go:21] Begin to register twin db model
  4. Oct 10 16:09:23 node1 edgecore[241646]: I1010 16:09:23.205673  241646 module.go:52] Module twin registered successfully
  5. Oct 10 16:09:23 node1 edgecore[241646]: I1010 16:09:23.228239  241646 client.go:78] "Connecting to docker on the dockerEndpoint" endpoint="unix:///var/run/docker.sock"
  6. Oct 10 16:09:23 node1 edgecore[241646]: I1010 16:09:23.228365  241646 client.go:97] "Start docker client with request timeout" timeout="0s"
  7. Oct 10 16:09:23 node1 edgecore[241646]: I1010 16:09:23.262259  241646 docker_service.go:242] "Hairpin mode is set" hairpinMode=hairpin-veth
  8. Oct 10 16:09:23 node1 edgecore[241646]: I1010 16:09:23.263310  241646 cni.go:204] "Error validating CNI config list" configList="{\n  \"name\": \"cbr0\",\n  \"cniVersion\": \"0.3.1\",\n  \"plugins\": [\n    {\n      \"type\": \"flannel\",\n     >
  9. Oct 10 16:09:23 node1 edgecore[241646]: I1010 16:09:23.263411  241646 cni.go:239] "Unable to update cni config" err="no valid networks found in /etc/cni/net.d"
  10. Oct 10 16:09:23 node1 edgecore[241646]: I1010 16:09:23.269385  241646 hostport_manager.go:72] "The binary conntrack is not installed, this can cause failures in network connection cleanup."
  11. Oct 10 16:09:23 node1 edgecore[241646]: I1010 16:09:23.269521  241646 hostport_manager.go:72] "The binary conntrack is not installed, this can cause failures in network connection cleanup."
  12. Oct 10 16:09:23 node1 edgecore[241646]: I1010 16:09:23.275158  241646 docker_service.go:257] "Docker cri networking managed by the network plugin" networkPluginName="kubernetes.io/no-op"
  13. Oct 10 16:09:23 node1 edgecore[241646]: I1010 16:09:23.312914  241646 docker_service.go:263] "Docker Info" dockerInfo=&{ID:X4KE:OB4J:5JH2:IYG4:X2FR:G4LB:OOWP:FUNK:6L46:7LM5:7CD4:ONYQ Containers:0 ContainersRunning:0 ContainersPaused:0 Containers>
  14. Oct 10 16:09:23 node1 edgecore[241646]: I1010 16:09:23.313017  241646 docker_service.go:278] "Setting cgroupDriver" cgroupDriver="cgroupfs"
  15. Oct 10 16:09:23 node1 edgecore[241646]: I1010 16:09:23.313345  241646 edged.go:716] RemoteRuntimeEndpoint: "unix:///var/run/dockershim.sock", remoteImageEndpoint: "unix:///var/run/dockershim.sock"
  16. Oct 10 16:09:23 node1 edgecore[241646]: I1010 16:09:23.313386  241646 edged.go:719] Starting the GRPC server for the docker CRI shim.
  17. Oct 10 16:09:23 node1 edgecore[241646]: I1010 16:09:23.361424  241646 state_mem.go:36] "Initialized new in-memory state store"
  18. Oct 10 16:09:23 node1 edgecore[241646]: W1010 16:09:23.361691  241646 edged.go:416] failed to get the edged's cgroup: mountpoint for memory not found. Edged system container metrics may be missing.
  19. Oct 10 16:09:23 node1 edgecore[241646]: W1010 16:09:23.361945  241646 edged.go:423] failed to get the container runtime's cgroup: failed to get container name for docker process: mountpoint for memory not found. Runtime system container metrics >
  20. Oct 10 16:09:23 node1 edgecore[241646]: W1010 16:09:23.560195  241646 sysinfo.go:203] Nodes topology is not available, providing CPU topology
  21. Oct 10 16:09:23 node1 edgecore[241646]: I1010 16:09:23.670596  241646 kuberuntime_manager.go:245] "Container runtime initialized" containerRuntime="docker" version="19.03.12" apiVersion="1.40.0"
  22. Oct 10 16:09:23 node1 edgecore[241646]: I1010 16:09:23.670736  241646 edged.go:600] --cgroups-per-qos enabled, but --cgroup-root was not specified.  defaulting to /
  23. Oct 10 16:09:23 node1 edgecore[241646]: I1010 16:09:23.693689  241646 container_manager_linux.go:280] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
  24. Oct 10 16:09:23 node1 edgecore[241646]: I1010 16:09:23.693873  241646 container_manager_linux.go:285] "Creating Container Manager object based on Node Config" nodeConfig={RuntimeCgroupsName: SystemCgroupsName: KubeletCgroupsName: ContainerRuntim>
  25. Oct 10 16:09:23 node1 edgecore[241646]: I1010 16:09:23.693953  241646 topology_manager.go:133] "Creating topology manager with policy per scope" topologyPolicyName="none" topologyScopeName="container"
  26. Oct 10 16:09:23 node1 edgecore[241646]: I1010 16:09:23.694000  241646 container_manager_linux.go:320] "Creating device plugin manager" devicePluginEnabled=false
  27. Oct 10 16:09:23 node1 edgecore[241646]: I1010 16:09:23.694039  241646 state_mem.go:36] "Initialized new in-memory state store"
  28. Oct 10 16:09:23 node1 edgecore[241646]: I1010 16:09:23.694294  241646 module.go:52] Module edged registered successfully
  29. Oct 10 16:09:23 node1 edgecore[241646]: I1010 16:09:23.694335  241646 module.go:52] Module websocket registered successfully
  30. Oct 10 16:09:23 node1 edgecore[241646]: I1010 16:09:23.694357  241646 module.go:52] Module eventbus registered successfully
  31. Oct 10 16:09:23 node1 edgecore[241646]: I1010 16:09:23.694512  241646 metamanager.go:47] Begin to register metaManager db model
  32. Oct 10 16:09:23 node1 edgecore[241646]: I1010 16:09:23.694691  241646 module.go:52] Module metaManager registered successfully
  33. Oct 10 16:09:23 node1 edgecore[241646]: W1010 16:09:23.694721  241646 module.go:55] Module servicebus is disabled, do not register
  34. Oct 10 16:09:23 node1 edgecore[241646]: W1010 16:09:23.694768  241646 module.go:55] Module edgestream is disabled, do not register
  35. Oct 10 16:09:23 node1 edgecore[241646]: W1010 16:09:23.694791  241646 module.go:55] Module testManager is disabled, do not register
  36. Oct 10 16:09:23 node1 edgecore[241646]: table `device` already exists, skip
  37. Oct 10 16:09:23 node1 edgecore[241646]: table `device_attr` already exists, skip
  38. Oct 10 16:09:23 node1 edgecore[241646]: table `device_twin` already exists, skip
  39. Oct 10 16:09:23 node1 edgecore[241646]: table `sub_topics` already exists, skip
  40. Oct 10 16:09:23 node1 edgecore[241646]: table `meta` already exists, skip
  41. Oct 10 16:09:23 node1 edgecore[241646]: table `meta_v2` already exists, skip
  42. Oct 10 16:09:23 node1 edgecore[241646]: table `target_urls` already exists, skip
  43. Oct 10 16:09:23 node1 edgecore[241646]: I1010 16:09:23.698655  241646 core.go:46] starting module twin
  44. Oct 10 16:09:23 node1 edgecore[241646]: I1010 16:09:23.698757  241646 core.go:46] starting module edged
  45. Oct 10 16:09:23 node1 edgecore[241646]: I1010 16:09:23.698828  241646 process.go:114] Begin to sync sqlite
  46. Oct 10 16:09:23 node1 edgecore[241646]: I1010 16:09:23.698881  241646 core.go:46] starting module websocket
  47. Oct 10 16:09:23 node1 edgecore[241646]: I1010 16:09:23.698889  241646 edged.go:303] Starting edged...
  48. Oct 10 16:09:23 node1 edgecore[241646]: I1010 16:09:23.698968  241646 core.go:46] starting module eventbus
  49. Oct 10 16:09:23 node1 edgecore[241646]: I1010 16:09:23.699042  241646 core.go:46] starting module metaManager
  50. Oct 10 16:09:23 node1 edgecore[241646]: I1010 16:09:23.699182  241646 http.go:40] tlsConfig InsecureSkipVerify true
  51. Oct 10 16:09:23 node1 edgecore[241646]: E1010 16:09:23.699481  241646 csi_plugin.go:222] kubernetes.io/csi: CSIDriverLister not found on KubeletVolumeHost
  52. Oct 10 16:09:23 node1 edgecore[241646]: I1010 16:09:23.699585  241646 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
  53. Oct 10 16:09:23 node1 edgecore[241646]: I1010 16:09:23.699596  241646 common.go:97] start connect to mqtt server with client id: hub-client-sub-1665389363
  54. Oct 10 16:09:23 node1 edgecore[241646]: I1010 16:09:23.699921  241646 common.go:99] client hub-client-sub-1665389363 isconnected: false
  55. Oct 10 16:09:23 node1 edgecore[241646]: I1010 16:09:23.705799  241646 client.go:157] finish hub-client sub
  56. Oct 10 16:09:23 node1 edgecore[241646]: I1010 16:09:23.705943  241646 common.go:97] start connect to mqtt server with client id: hub-client-pub-1665389363
  57. Oct 10 16:09:23 node1 edgecore[241646]: I1010 16:09:23.705970  241646 common.go:99] client hub-client-pub-1665389363 isconnected: false
  58. Oct 10 16:09:23 node1 edgecore[241646]: I1010 16:09:23.706533  241646 client.go:95] edge-hub-cli subscribe topic to $hw/events/upload/#
  59. Oct 10 16:09:23 node1 edgecore[241646]: I1010 16:09:23.707352  241646 client.go:176] finish hub-client pub
  60. Oct 10 16:09:23 node1 edgecore[241646]: I1010 16:09:23.707411  241646 eventbus.go:69] Init Sub And Pub Client for external mqtt broker tcp://127.0.0.1:1883 successfully
  61. Oct 10 16:09:23 node1 edgecore[241646]: I1010 16:09:23.707714  241646 client.go:95] edge-hub-cli subscribe topic to $hw/events/device/+/state/update
  62. Oct 10 16:09:23 node1 edgecore[241646]: I1010 16:09:23.708535  241646 client.go:95] edge-hub-cli subscribe topic to $hw/events/device/+/twin/+
  63. Oct 10 16:09:23 node1 edgecore[241646]: I1010 16:09:23.709211  241646 client.go:95] edge-hub-cli subscribe topic to $hw/events/node/+/membership/get
  64. Oct 10 16:09:23 node1 edgecore[241646]: I1010 16:09:23.709968  241646 client.go:95] edge-hub-cli subscribe topic to SYS/dis/upload_records
  65. Oct 10 16:09:23 node1 edgecore[241646]: I1010 16:09:23.710532  241646 client.go:95] edge-hub-cli subscribe topic to +/user/#
  66. Oct 10 16:09:23 node1 edgecore[241646]: I1010 16:09:23.711463  241646 client.go:103] list edge-hub-cli-topics status, no record, skip sync
  67. Oct 10 16:09:24 node1 edgecore[241646]: I1010 16:09:24.162840  241646 cpu_manager.go:209] "Starting CPU manager" policy="none"
  68. Oct 10 16:09:24 node1 edgecore[241646]: I1010 16:09:24.162909  241646 cpu_manager.go:210] "Reconciling" reconcilePeriod="0s"
  69. Oct 10 16:09:24 node1 edgecore[241646]: I1010 16:09:24.162984  241646 state_mem.go:36] "Initialized new in-memory state store"
  70. Oct 10 16:09:24 node1 edgecore[241646]: I1010 16:09:24.163639  241646 state_mem.go:88] "Updated default CPUSet" cpuSet=""
  71. Oct 10 16:09:24 node1 edgecore[241646]: I1010 16:09:24.163709  241646 state_mem.go:96] "Updated CPUSet assignments" assignments=map[]
  72. Oct 10 16:09:24 node1 edgecore[241646]: I1010 16:09:24.163743  241646 policy_none.go:49] "None policy: Start"
  73. Oct 10 16:09:24 node1 edgecore[241646]: I1010 16:09:24.167587  241646 memory_manager.go:168] "Starting memorymanager" policy="None"
  74. Oct 10 16:09:24 node1 edgecore[241646]: I1010 16:09:24.167715  241646 state_mem.go:35] "Initializing new in-memory state store"
  75. Oct 10 16:09:24 node1 edgecore[241646]: I1010 16:09:24.168271  241646 state_mem.go:75] "Updated machine memory state"
  76. Oct 10 16:09:24 node1 edgecore[241646]: E1010 16:09:24.169212  241646 edged.go:824] Failed to start container manager, err: system validation failed - Following Cgroup subsystem not mounted: [memory]
  77. Oct 10 16:09:24 node1 edgecore[241646]: E1010 16:09:24.169265  241646 edged.go:307] initialize module error: system validation failed - Following Cgroup subsystem not mounted: [memory]
  78. Oct 10 16:09:24 node1 systemd[1]: edgecore.service: Main process exited, code=exited, status=1/FAILURE

查看错误日志

  1. Oct 10 21:40:05 node1 edgecore[466671]: I1010 21:40:05.212882  466671 server.go:76] Version: v1.10.3
  2. Oct 10 21:40:05 node1 edgecore[466671]: I1010 21:40:05.225448  466671 server.go:103] Use node IP address from config: 192.168.137.74
  3. Oct 10 21:40:05 node1 edgecore[466671]: I1010 21:40:05.225600  466671 sql.go:21] Begin to register twin db model
  4. Oct 10 21:40:05 node1 edgecore[466671]: I1010 21:40:05.226142  466671 module.go:52] Module twin registered successfully
  5. Oct 10 21:40:05 node1 edgecore[466671]: I1010 21:40:05.249774  466671 client.go:78] "Connecting to docker on the dockerEndpoint" endpoint="unix:///var/run/docker.sock"
  6. Oct 10 21:40:05 node1 edgecore[466671]: I1010 21:40:05.249920  466671 client.go:97] "Start docker client with request timeout" timeout="0s"
  7. Oct 10 21:40:05 node1 edgecore[466671]: I1010 21:40:05.284696  466671 docker_service.go:242] "Hairpin mode is set" hairpinMode=hairpin-veth
  8. Oct 10 21:40:05 node1 edgecore[466671]: I1010 21:40:05.285558  466671 cni.go:204] "Error validating CNI config list" configList="{\n  \"name\": \"cbr0\",\n  \"cniVersion\": \"0.3.1\",\n  \"plugins\": [\n    {\n      \"type\": \"flannel\",\n      \"delegate\": {\n        \"hairpinMode\": true,\n        \"isDefaultGateway\": true\n      }\n    >
  9. Oct 10 21:40:05 node1 edgecore[466671]: I1010 21:40:05.285634  466671 cni.go:239] "Unable to update cni config" err="no valid networks found in /etc/cni/net.d"
  10. Oct 10 21:40:05 node1 edgecore[466671]: I1010 21:40:05.291651  466671 hostport_manager.go:72] "The binary conntrack is not installed, this can cause failures in network connection cleanup."
  11. Oct 10 21:40:05 node1 edgecore[466671]: I1010 21:40:05.291816  466671 hostport_manager.go:72] "The binary conntrack is not installed, this can cause failures in network connection cleanup."
  12. Oct 10 21:40:05 node1 edgecore[466671]: I1010 21:40:05.298601  466671 docker_service.go:257] "Docker cri networking managed by the network plugin" networkPluginName="kubernetes.io/no-op"
  13. Oct 10 21:40:05 node1 edgecore[466671]: I1010 21:40:05.337487  466671 docker_service.go:263] "Docker Info" dockerInfo=&{ID:X4KE:OB4J:5JH2:IYG4:X2FR:G4LB:OOWP:FUNK:6L46:7LM5:7CD4:ONYQ Containers:0 ContainersRunning:0 ContainersPaused:0 ContainersStopped:0 Images:0 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] >
  14. Oct 10 21:40:05 node1 edgecore[466671]: E1010 21:40:05.337737  466671 edged.go:279] init new edged error, misconfiguration: kubelet cgroup driver: "systemd" is different from docker cgroup driver: "cgroupfs"

vim /etc/docker/daemon.jsn

# 把native.cgroupdriver=systemd 修改成 native.cgroupdriver=cgroupfs

  1. docker info|grep "Cgroup Driver" 
  2.  
  3. #检查 是否输出 Cgroup Driver: cgroupfs

#重启docker 并且查看docker的状态

systemctl restart docker

systemctl status docker

docker 运行正常

vim /var/lib/kubelet/kubeadm-flags.env

把其中的systemd 也改成cgroupfs再启动下kubelet，重新载入配置

  1. systemctl daemon-reload
  2. systemctl restart kubelet
  3. systemctl enable kubelet
  4. systemctl start kubelet
  5. systemctl status kubelet

查看日志发现 

  1. level=warning msg="Your kernel does not support cgroup rt runtime"
  2. master dockerd[22355]: time="2022-10-10T21:51:48.629194761+08:00" level=warning msg="Your kernel does not support cgroup blkio weight"
  3. master dockerd[22355]: time="2022-10-10T21:51:48.629205446+08:00" level=warning msg="Your kernel does not support cgroup blkio weight_device"

主节点docker，此条警告可以忽略，不影响。

再次加入kubeedge，发现报错

  1. Failed to start container manager, err: system validation failed - Following Cgroup subsystem not mounted: [memory]
  2. 2388 edged.go:307] initialize module error: system validation failed - Following Cgroup subsystem not mounted: [memory]
  3. ess exited, code=exited, status=1/FAILURE

日志显示的是内存资源的cgroups机制没有开启。
查看可用的cgroup:

  1. root@node1:~/keadm-v1.10.3-linux-arm64/keadm# cat /proc/cgroups
  2. #subsys_name    hierarchy       num_cgroups     enabled
  3. cpuset  8       1       1
  4. cpu     2       102     1
  5. cpuacct 2       102     1
  6. blkio   7       102     1
  7. memory  0       112     0
  8. devices 6       102     1
  9. freezer 10      2       1
  10. net_cls 3       1       1
  11. perf_event      11      1       1
  12. net_prio        3       1       1
  13. hugetlb 9       1       1
  14. pids    4       111     1
  15. rdma    5       1       1

修改启动参数

sudo vim /boot/firmware/cmdline.txt

添加以下内容：

cgroup_memory=1 cgroup_enable=memory

添加在同一行的最后面,接着内容后空格后添加, 不要换行

#重启机器配置生效

reboot 

  1. #查看edgecore启动情况
  2. ps aux|grep edgecore
  3. root@node1:~/keadm-v1.10.3-linux-arm64/keadm# ps aux|grep edgecore
  4. root         697  5.4  1.9 1657340 77492 ?       Ssl  23:00   0:52 /usr/local/bin/edgecore
  5. root        2563  0.0  0.0   7696   744 pts/0    S+   23:16   0:00 grep --color=auto edgecore
  6.  
  7. #查看启动状态
  8. sudo systemctl status edgecore
  9.  
  10.  edgecore.service
  11.      Loaded: loaded (/etc/systemd/system/edgecore.service; bad; vendor preset: enabled)
  12.      Active: active (running) since Mon 2022-10-10 23:00:25 CST; 16min ago
  13.    Main PID: 697 (edgecore)
  14.       Tasks: 13 (limit: 4435)
  15.      Memory: 88.4M
  16.      CGroup: /system.slice/edgecore.service
  17.              └─697 /usr/local/bin/edgecore
  18.  
  19. #查看docker开机启动状态 enabled:开启, disabled:关闭
  20. systemctl is-enabled edgecore
  21.  
  22. #设置开机自启动
  23. systemctl enable edgecore

Ubuntu18.04+树莓派Ubuntu20.04部署kubeedge v1.10.3主要参考帖子

(39条消息) Ubuntu20.04+树莓派部署Kubeedge v1.6.1_YsyNoobCoder的博客-CSDN博客_kubeedge 树莓派(39条消息) KubeEdge 超详细部署记录&问题记载6.28_Jason-Ning的博客-CSDN博客

验证环节

(39条消息) K8S v1.17.17&KubeEdge v1.7部署指南+kubeedge-counter-demo示例_cike90的博客-CSDN博客

  1. 常用指令
  2. #查看 systemd托管列表
  3. systemctl list-unit-files
  4. systemctl start cloudcore.service && systemctl enable cloudcore.service
  5.  
  6. #查看节点状态
  7. kubectl get node
  8. kubectl get po -A
  9. kubectl get po -n kube-system
  10. kubectl get po -o wide
  11.  
  12. #查看网络状态
  13. yum install net-tools -y
  14. netstat -ntlp
  15.  
  16. #查看docker状态docker
  17. docker images | grep kubeedge-pi-counter
  18.  
  19. #传文件
  20. scp kubeedge-pi-counter.tar 192.168.xx.xx:/root/
  21. tmp目录下的文件
  22.  
  23. 注意事项：
  24. 1、一般情况下，使用完成尽量挂起虚拟机，不然容器停掉，启动后node会处于not reday状态。
  25. 2、如果node处于not reday状态了，执行一下重启脚本，执行命令如下：
  26.  
  27. bash restart_kubeedge.sh
  28.  
  29. 脚本内容：
  30. #!/bin/bash
  31. for i in `docker ps -a | awk '{print $1}' | grep -v CONT`;do docker restart $i;done
  32. for i in `docker ps -a | awk '{print $1}' | grep -v CONT`;do docker restart $i;done

 (39条消息) 【云原生1】零基础 k8s + kubeedge v1.5.0 部署_烈日松饼的博客-CSDN博客_kubeedge和k8s

（主要） 

 KubeEdge 部署记录 | 生命不息 折腾不止 (konata.tech)

排除错误：

kubernetes.io/csi: CSIDriverLister not found on KubeletVolumeHost

 (39条消息) KubeEdge1.10从零开始详细搭建教程_孟诗远msy的博客-CSDN博客_kubeedge搭建

声明：本文内容由网友自发贡献，转载请注明出处：【wpsshop博客】
推荐阅读
相关标签

Copyright © 2003-2013 www.wpsshop.cn 版权所有，并保留所有权利。

  

闽ICP备14008679号