AttributeError: module 'torch.nn.init' has no attribute 'zeros_'

秒速五厘米 2022-03-17 09:05 518阅读 0赞

# 问题 #

File "E:\wj-lab\expStad\model.py", line 50, in __init__
        feat_block.apply(weights_init_kaiming)
      File "D:\Anaconda3\envs\python35\lib\site-packages\torch\nn\modules\module.py", line 231, in apply
        module.apply(fn)
    
      File "D:\Anaconda3\envs\python35\lib\site-packages\torch\nn\modules\module.py", line 232, in apply
        fn(self)
      File "E:\wj-lab\expStad\model.py", line 26, in weights_init_kaiming
        init.zeros(m.bias.data)
    
    AttributeError: module 'torch.nn.init' has no attribute 'zeros_'

# 出错源码 #

def weights_init_kaiming(m):
      classname = m.__class__.__name__  
      if classname.find('Conv') != -1:
            init.kaiming_normal_(m.weight.data, a=0, mode='fan_in')
    
        elif classname.find('Linear') != -1:
            init.kaiming_normal_(m.weight.data, a=0, mode='fan_out')
            init.zeros(m.bias.data)
        
        elif classname.find('BatchNorm1d') != -1:
            init.normal_(m.weight.data, 1.0, 0.01)
            init.zeros(m.bias.data)
    
    def weights_init_classifier(m):
        classname = m.__class__.__name__
        if classname.find('Linear') != -1:
            init.normal_(m.weight.data, 0, 0.001)
            init.zeros(m.bias.data)

# 原因 #

>>> python3   # 进入python 环境
    >>> import torch
    >>> dir(torch)  # 查看pytorch支持环境，是否有属性'__init_'
     
    如下：
    [ ByteStorage', 'ByteTensor',  'CharTensor', 'CompiledFunction',  'CudaCharStorageBase', 'CudaDoubleStorageBase', 'CudaFloatStorageBase', 'CudaIntStorageBase', 'CudaLongStorageBase '__init__', '__le__', '__loader__' ........]
    
    >>> dir(torch.nn.init) # 查看torch.nn.init 支持环境，是否有'attribute zeros_'
    
    如下：
    ['__builtins__','__cached__','__doc__','__file__','__loader__','__package__','kaiming_normal', 'kaiming_normal_', 'kaiming_uniform', 'kaiming_uniform_', 'math', 'normal', 'normal_'   ......]

发现根本没有 'attribute zeros\_'  说明错误原因

1.  源码有问题，源码版本是python2
2.  当前python环境缺少库函数，使得不支持 attribute \#\#

pytorch 的 torch.nn.init 中文文档

[https://pytorch-cn.readthedocs.io/zh/latest/package\_references/nn\_init/][https_pytorch-cn.readthedocs.io_zh_latest_package_references_nn_init]

torch.nn.init.kaiming\_normal() Example 中文文档

[https://www.programcreek.com/python/example/108248/torch.nn.init.kaiming\_normal][https_www.programcreek.com_python_example_108248_torch.nn.init.kaiming_normal]

# 解决办法 #

def weights_init_kaiming(m):
        classname = m.__class__.__name__
        if classname.find('Conv') != -1:
            m.weight.data = init.kaiming_normal(m.weight.data, a=0, mode='fan_in')
           
        elif classname.find('Linear') != -1:      
            m.weight.data = init.kaiming_normal(m.weight.data, a=0, mode='fan_out')
            m.bias.data.fill_(0)
          
        elif classname.find('BatchNorm1d') != -1:
            m.weight.data.normal_(1.0, 0.01)
            m.bias.data.fill_(0)
    
    
    def weights_init_classifier(m):
        classname = m.__class__.__name__
        if classname.find('Linear') != -1:
            m.weight.data = init.normal_(m.weight.data,  0, 0.001)
            m.bias.data.fill_(0)

[https_pytorch-cn.readthedocs.io_zh_latest_package_references_nn_init]: https://pytorch-cn.readthedocs.io/zh/latest/package_references/nn_init/
[https_www.programcreek.com_python_example_108248_torch.nn.init.kaiming_normal]: https://www.programcreek.com/python/example/108248/torch.nn.init.kaiming_normal