devbox
程序代码艺术家
这个屌丝很懒，什么也没留下！
关注作者
热门标签
热门文章
当前位置:   article > 正文

让keras训练深度网络时使用多个显卡_keras_model.fit 设置多张卡

作者：程序代码艺术家 | 2024-02-01 12:26:41

赞

踩

keras_model.fit 设置多张卡

1、使用nvidia-smi pmon 查看linux系统的gpu情况，如下：


显然是2张显卡，如何让它们都工作呢

2、keras提供了keras.utils import multi_gpu_model使用多个显卡的功能：

在原来的model基础上使用multi_gpu_model函数指定一下gpu个数即可：

model =  multi_gpu_model(model, 2)


完整列子如下（如粗黑色字体描述）：

root@deeplearning:/opt/soft/keras/examples# cat mnist_mlp_multi_gpu.py       

  1. '''Trains a simple deep NN on the MNIST dataset.
  3. Gets to 98.40% test accuracy after 20 epochs
  4. (there is *a lot* of margin for parameter tuning).
  5. 2 seconds per epoch on a K520 GPU.
  6. '''
  7.  
  8. from __future__ import print_function
  9.  
  10. import keras
  11. from keras.datasets import mnist
  12. from keras.models import Sequential
  13. from keras.layers import Dense, Dropout
  14. from keras.optimizers import RMSprop
  15. from keras.utils import multi_gpu_model
  16.  
  17. batch_size = 128
  18. num_classes = 10
  19. epochs = 20
  20.  
  21. # the data, shuffled and split between train and test sets
  22. (x_train, y_train), (x_test, y_test) = mnist.load_data()
  23.  
  24. x_train = x_train.reshape(60000, 784)
  25. x_test = x_test.reshape(10000, 784)
  26. x_train = x_train.astype('float32')
  27. x_test = x_test.astype('float32')
  28. x_train /= 255
  29. x_test /= 255
  30. print(x_train.shape[0], 'train samples')
  31. print(x_test.shape[0], 'test samples')
  32.  
  33.  
  34. # convert class vectors to binary class matrices
  35. y_train = keras.utils.to_categorical(y_train, num_classes)
  36. y_test = keras.utils.to_categorical(y_test, num_classes)
  37.  
  38.  
  39. model = Sequential()
  40. model.add(Dense(512, activation='relu', input_shape=(784,)))
  41. model.add(Dropout(0.2))
  42. model.add(Dense(512, activation='relu'))
  43. model.add(Dropout(0.2))
  44. model.add(Dense(num_classes, activation='softmax'))
  45.  
  46.  
  47. model = multi_gpu_model(model, 2)
  48. model.summary()
  49.  
  50.  
  51. model.compile(loss='categorical_crossentropy',
  52.               optimizer=RMSprop(),
  53.               metrics=['accuracy'])
  54.  
  55.  
  56. history = model.fit(x_train, y_train,
  57.                     batch_size=batch_size,
  58.                     epochs=epochs,
  59.                     verbose=1,
  60.                     validation_data=(x_test, y_test))
  61. score = model.evaluate(x_test, y_test, verbose=0)
  62. print('Test loss:', score[0])
  63. print('Test accuracy:', score[1])

最终运行的结果如下：

  1. root@deeplearning:/opt/soft/keras/examples# python mnist_mlp_multi_gpu.py 
  2. Using TensorFlow backend.
  3. 60000 train samples
  4. 10000 test samples
  5. 2018-02-17 18:14:41.147652: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
  6. 2018-02-17 18:14:41.304376: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:892] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
  7. 2018-02-17 18:14:41.304757: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1030] Found device 0 with properties: 
  8. name: Tesla P4 major: 6 minor: 1 memoryClockRate(GHz): 1.1135
  9. pciBusID: 0000:00:07.0
  10. totalMemory: 7.43GiB freeMemory: 7.32GiB
  11. 2018-02-17 18:14:41.399113: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:892] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
  12. 2018-02-17 18:14:41.399487: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1030] Found device 1 with properties: 
  13. name: Tesla P4 major: 6 minor: 1 memoryClockRate(GHz): 1.1135
  14. pciBusID: 0000:00:08.0
  15. totalMemory: 7.43GiB freeMemory: 7.32GiB
  16. 2018-02-17 18:14:41.399579: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1045] Device peer to peer matrix
  17. 2018-02-17 18:14:41.399613: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1051] DMA: 0 1 
  18. 2018-02-17 18:14:41.399627: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1061] 0:   Y N 
  19. 2018-02-17 18:14:41.399632: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1061] 1:   N Y 
  20. 2018-02-17 18:14:41.399650: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1120] Creating TensorFlow device (/device:GPU:0) -> (device: 0, name: Tesla P4, pci bus id: 0000:00:07.0, compute capability: 6.1)
  21. 2018-02-17 18:14:41.399663: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1120] Creating TensorFlow device (/device:GPU:1) -> (device: 1, name: Tesla P4, pci bus id: 0000:00:08.0, compute capability: 6.1)
  22. __________________________________________________________________________________________________
  23. Layer (type)                    Output Shape         Param #     Connected to                     
  24. ==================================================================================================
  25. dense_1_input (InputLayer)      (None, 784)          0                                            
  26. __________________________________________________________________________________________________
  27. lambda_1 (Lambda)               (None, 784)          0           dense_1_input[0][0]              
  28. __________________________________________________________________________________________________
  29. lambda_2 (Lambda)               (None, 784)          0           dense_1_input[0][0]              
  30. __________________________________________________________________________________________________
  31. sequential_1 (Sequential)       (None, 10)           669706      lambda_1[0][0]                   
  32.                                                                  lambda_2[0][0]                   
  33. __________________________________________________________________________________________________
  34. dense_3 (Concatenate)           (None, 10)           0           sequential_1[1][0]               
  35.                                                                  sequential_1[2][0]               
  36. ==================================================================================================
  37. Total params: 669,706
  38. Trainable params: 669,706
  39. Non-trainable params: 0
  40. __________________________________________________________________________________________________
  41. Train on 60000 samples, validate on 10000 samples
  42. Epoch 1/20
  43. 60000/60000 [==============================] - 4s 61us/step - loss: 0.2460 - acc: 0.9234 - val_loss: 0.1110 - val_acc: 0.9647
  44. Epoch 2/20
  45. 60000/60000 [==============================] - 3s 49us/step - loss: 0.1018 - acc: 0.9693 - val_loss: 0.0804 - val_acc: 0.9766
  46. Epoch 3/20
  47. 60000/60000 [==============================] - 3s 49us/step - loss: 0.0742 - acc: 0.9776 - val_loss: 0.0745 - val_acc: 0.9771
  48. Epoch 4/20
  49. 60000/60000 [==============================] - 3s 49us/step - loss: 0.0599 - acc: 0.9820 - val_loss: 0.0698 - val_acc: 0.9793
  50. Epoch 5/20
  51. 60000/60000 [==============================] - 3s 49us/step - loss: 0.0514 - acc: 0.9844 - val_loss: 0.0761 - val_acc: 0.9793
  52. Epoch 6/20
  53. 60000/60000 [==============================] - 3s 49us/step - loss: 0.0425 - acc: 0.9872 - val_loss: 0.0775 - val_acc: 0.9800
  54. Epoch 7/20
  55. 60000/60000 [==============================] - 3s 49us/step - loss: 0.0390 - acc: 0.9886 - val_loss: 0.0840 - val_acc: 0.9813
  56. Epoch 8/20
  57. 60000/60000 [==============================] - 3s 49us/step - loss: 0.0334 - acc: 0.9903 - val_loss: 0.0890 - val_acc: 0.9805
  58. Epoch 9/20
  59. 60000/60000 [==============================] - 3s 49us/step - loss: 0.0310 - acc: 0.9904 - val_loss: 0.0909 - val_acc: 0.9818
  60. Epoch 10/20
  61. 60000/60000 [==============================] - 3s 49us/step - loss: 0.0277 - acc: 0.9920 - val_loss: 0.0877 - val_acc: 0.9825
  62. Epoch 11/20
  63. 60000/60000 [==============================] - 3s 49us/step - loss: 0.0264 - acc: 0.9924 - val_loss: 0.0905 - val_acc: 0.9832
  64. Epoch 12/20
  65. 60000/60000 [==============================] - 3s 49us/step - loss: 0.0257 - acc: 0.9926 - val_loss: 0.0868 - val_acc: 0.9836
  66. Epoch 13/20
  67. 60000/60000 [==============================] - 3s 49us/step - loss: 0.0232 - acc: 0.9936 - val_loss: 0.0944 - val_acc: 0.9837
  68. Epoch 14/20
  69. 60000/60000 [==============================] - 3s 49us/step - loss: 0.0217 - acc: 0.9941 - val_loss: 0.1022 - val_acc: 0.9834
  70. Epoch 15/20
  71. 60000/60000 [==============================] - 3s 49us/step - loss: 0.0223 - acc: 0.9938 - val_loss: 0.0952 - val_acc: 0.9816
  72. Epoch 16/20
  73. 60000/60000 [==============================] - 3s 49us/step - loss: 0.0190 - acc: 0.9949 - val_loss: 0.1015 - val_acc: 0.9840
  74. Epoch 17/20
  75. 60000/60000 [==============================] - 3s 49us/step - loss: 0.0197 - acc: 0.9946 - val_loss: 0.1161 - val_acc: 0.9848
  76. Epoch 18/20
  77. 60000/60000 [==============================] - 3s 49us/step - loss: 0.0210 - acc: 0.9945 - val_loss: 0.1078 - val_acc: 0.9822
  78. Epoch 19/20
  79. 60000/60000 [==============================] - 3s 49us/step - loss: 0.0190 - acc: 0.9950 - val_loss: 0.1235 - val_acc: 0.9832
  80. Epoch 20/20
  81. 60000/60000 [==============================] - 3s 49us/step - loss: 0.0185 - acc: 0.9954 - val_loss: 0.0980 - val_acc: 0.9843
  82. Test loss: 0.0980480428516
  83. Test accuracy: 0.9843

声明：本文内容由网友自发贡献，不代表【wpsshop博客】立场，版权归原作者所有，本站不承担相应法律责任。如您发现有侵权的内容，请联系我们。转载请注明出处：https://www.wpsshop.cn/article/detail/53775
推荐阅读
相关标签

Copyright © 2003-2013 www.wpsshop.cn 版权所有，并保留所有权利。

  

闽ICP备14008679号