tensorflow报错 'failed to create cublas handle', 'Failed to get convolution algorithm'

发表于 分类于 本文字数： 146 阅读时长 ≈ 1 分钟

tensorflow gpu 报错：

1
2
3
4
5
... tensorflow/stream_executor/cuda/cuda_blas.cc:238] failed to create cublas handle: CUBLAS_STATUS_ALLOC_FAILED
...
... tensorflow/stream_executor/cuda/cuda_dnn.cc:329] Could not create cudnn handle: CUDNN_STATUS_ALLOC_FAILED
...
... Unknown:  Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above.

软件版本：

  • python: 3.7.7
  • keras: 2.3.1
  • tensorboard: 2.1.0
  • tensorflow: 2.1.0

显卡版本：

  • gpu: RTX 2070
  • cuda: 10.1
  • cudnn: 7.8

解决办法：

You may try limiting your gpu memory usage with set_memory_growth option.
See https://www.tensorflow.org/guide/gpu#limiting_gpu_memory_growth
Put the following snippet on top of your code and execute again

1
2
3
import tensorflow as tf
gpus= tf.config.experimental.list_physical_devices('GPU')
tf.config.experimental.set_memory_growth(gpus[0], True)