pytorch中的named_parameters()和parameters()
pytorch named_parameters()和parameters()
nn.Module
nn.Module里面關(guān)于參數(shù)有兩個很重要的屬性named_parameters()和parameters(),前者給出網(wǎng)絡(luò)層的名字和參數(shù)的迭代器,而后者僅僅是參數(shù)的迭代器。
import torchvision.models as models
model = models.resnet18()
for param in model.named_parameters():
print(param[0])
'''
conv1.weight
bn1.weight
bn1.bias
layer1.0.conv1.weight
layer1.0.bn1.weight
layer1.0.bn1.bias
layer1.0.conv2.weight
layer1.0.bn2.weight
layer1.0.bn2.bias
layer1.1.conv1.weight
layer1.1.bn1.weight
layer1.1.bn1.bias
layer1.1.conv2.weight
layer1.1.bn2.weight
layer1.1.bn2.bias
layer2.0.conv1.weight
layer2.0.bn1.weight
layer2.0.bn1.bias
layer2.0.conv2.weight
layer2.0.bn2.weight
layer2.0.bn2.bias
layer2.0.downsample.0.weight
layer2.0.downsample.1.weight
layer2.0.downsample.1.bias
layer2.1.conv1.weight
layer2.1.bn1.weight
layer2.1.bn1.bias
layer2.1.conv2.weight
layer2.1.bn2.weight
layer2.1.bn2.bias
layer3.0.conv1.weight
layer3.0.bn1.weight
layer3.0.bn1.bias
layer3.0.conv2.weight
layer3.0.bn2.weight
layer3.0.bn2.bias
layer3.0.downsample.0.weight
layer3.0.downsample.1.weight
layer3.0.downsample.1.bias
layer3.1.conv1.weight
layer3.1.bn1.weight
layer3.1.bn1.bias
layer3.1.conv2.weight
layer3.1.bn2.weight
layer3.1.bn2.bias
layer4.0.conv1.weight
layer4.0.bn1.weight
layer4.0.bn1.bias
layer4.0.conv2.weight
layer4.0.bn2.weight
layer4.0.bn2.bias
layer4.0.downsample.0.weight
layer4.0.downsample.1.weight
layer4.0.downsample.1.bias
layer4.1.conv1.weight
layer4.1.bn1.weight
layer4.1.bn1.bias
layer4.1.conv2.weight
layer4.1.bn2.weight
layer4.1.bn2.bias
fc.weight
fc.bias
'''模型參數(shù):named_parameters()、parameters()、state_dict()區(qū)別
torch中存在3個功能極其類似的方法,它們分別是model.parameters()、model.named_parameters()、model.state_dict(),
下面就具體來說說這三個函數(shù)的差異:
一、model.parameters()和model.named_parameters()差別
- named_parameters()返回的list中,每個元組(與list相似,只是數(shù)據(jù)不可修改)打包了2個內(nèi)容,分別是
layer-name和layer-param(網(wǎng)絡(luò)層的名字和參數(shù)的迭代器); - parameters()只有后者layer-param(參數(shù)的迭代器)
1、model.named_parameters()里的網(wǎng)絡(luò)層名字
import torchvision.models as models
model = models.resnet18()
for param_tuple in model.named_parameters():
name, param = param_tuple
print("name = ", name)
print("-" * 100)打印結(jié)果:
name = conv1.weight
----------------------------------------------------------------------------------------------------
name = bn1.weight
----------------------------------------------------------------------------------------------------
name = bn1.bias
----------------------------------------------------------------------------------------------------
name = layer1.0.conv1.weight
----------------------------------------------------------------------------------------------------
name = layer1.0.bn1.weight
----------------------------------------------------------------------------------------------------
name = layer1.0.bn1.bias
----------------------------------------------------------------------------------------------------
name = layer1.0.conv2.weight
----------------------------------------------------------------------------------------------------
name = layer1.0.bn2.weight
----------------------------------------------------------------------------------------------------
name = layer1.0.bn2.bias
----------------------------------------------------------------------------------------------------
name = layer1.1.conv1.weight
----------------------------------------------------------------------------------------------------
name = layer1.1.bn1.weight
----------------------------------------------------------------------------------------------------
name = layer1.1.bn1.bias
----------------------------------------------------------------------------------------------------
name = layer1.1.conv2.weight
----------------------------------------------------------------------------------------------------
name = layer1.1.bn2.weight
----------------------------------------------------------------------------------------------------
name = layer1.1.bn2.bias
----------------------------------------------------------------------------------------------------
name = layer2.0.conv1.weight
----------------------------------------------------------------------------------------------------
name = layer2.0.bn1.weight
----------------------------------------------------------------------------------------------------
name = layer2.0.bn1.bias
----------------------------------------------------------------------------------------------------
name = layer2.0.conv2.weight
----------------------------------------------------------------------------------------------------
name = layer2.0.bn2.weight
----------------------------------------------------------------------------------------------------
name = layer2.0.bn2.bias
----------------------------------------------------------------------------------------------------
name = layer2.0.downsample.0.weight
----------------------------------------------------------------------------------------------------
name = layer2.0.downsample.1.weight
----------------------------------------------------------------------------------------------------
name = layer2.0.downsample.1.bias
----------------------------------------------------------------------------------------------------
name = layer2.1.conv1.weight
----------------------------------------------------------------------------------------------------
name = layer2.1.bn1.weight
----------------------------------------------------------------------------------------------------
name = layer2.1.bn1.bias
----------------------------------------------------------------------------------------------------
name = layer2.1.conv2.weight
----------------------------------------------------------------------------------------------------
name = layer2.1.bn2.weight
----------------------------------------------------------------------------------------------------
name = layer2.1.bn2.bias
----------------------------------------------------------------------------------------------------
name = layer3.0.conv1.weight
----------------------------------------------------------------------------------------------------
name = layer3.0.bn1.weight
----------------------------------------------------------------------------------------------------
name = layer3.0.bn1.bias
----------------------------------------------------------------------------------------------------
name = layer3.0.conv2.weight
----------------------------------------------------------------------------------------------------
name = layer3.0.bn2.weight
----------------------------------------------------------------------------------------------------
name = layer3.0.bn2.bias
----------------------------------------------------------------------------------------------------
name = layer3.0.downsample.0.weight
----------------------------------------------------------------------------------------------------
name = layer3.0.downsample.1.weight
----------------------------------------------------------------------------------------------------
name = layer3.0.downsample.1.bias
----------------------------------------------------------------------------------------------------
name = layer3.1.conv1.weight
----------------------------------------------------------------------------------------------------
name = layer3.1.bn1.weight
----------------------------------------------------------------------------------------------------
name = layer3.1.bn1.bias
----------------------------------------------------------------------------------------------------
name = layer3.1.conv2.weight
----------------------------------------------------------------------------------------------------
name = layer3.1.bn2.weight
----------------------------------------------------------------------------------------------------
name = layer3.1.bn2.bias
----------------------------------------------------------------------------------------------------
name = layer4.0.conv1.weight
----------------------------------------------------------------------------------------------------
name = layer4.0.bn1.weight
----------------------------------------------------------------------------------------------------
name = layer4.0.bn1.bias
----------------------------------------------------------------------------------------------------
name = layer4.0.conv2.weight
----------------------------------------------------------------------------------------------------
name = layer4.0.bn2.weight
----------------------------------------------------------------------------------------------------
name = layer4.0.bn2.bias
----------------------------------------------------------------------------------------------------
name = layer4.0.downsample.0.weight
----------------------------------------------------------------------------------------------------
name = layer4.0.downsample.1.weight
----------------------------------------------------------------------------------------------------
name = layer4.0.downsample.1.bias
----------------------------------------------------------------------------------------------------
name = layer4.1.conv1.weight
----------------------------------------------------------------------------------------------------
name = layer4.1.bn1.weight
----------------------------------------------------------------------------------------------------
name = layer4.1.bn1.bias
----------------------------------------------------------------------------------------------------
name = layer4.1.conv2.weight
----------------------------------------------------------------------------------------------------
name = layer4.1.bn2.weight
----------------------------------------------------------------------------------------------------
name = layer4.1.bn2.bias
----------------------------------------------------------------------------------------------------
name = fc.weight
----------------------------------------------------------------------------------------------------
name = fc.bias
----------------------------------------------------------------------------------------------------Process finished with exit code 0
2、model.named_parameters()里的網(wǎng)絡(luò)層名字、參數(shù)
import torchvision.models as models
model = models.resnet18()
for param_tuple in model.named_parameters():
name, param = param_tuple
print("name = ", name)
print("-" * 100)
print("param_tuple = ", param_tuple)
print("*" * 200)打印結(jié)果:
C:\Program_Files_AI\Anaconda3531\python.exe C:/Users/Admin/OneDrive/WorkSpace_AI/0-基于知識庫的智能問答系統(tǒng)-華控智加/01-意圖識別/test.py
name = conv1.weight
----------------------------------------------------------------------------------------------------
param_tuple = ('conv1.weight', Parameter containing:
tensor([[[[-1.4115e-05, 2.9187e-02, 2.9325e-03, ..., -4.2247e-02,
1.7490e-02, -4.5253e-02],
[-2.4594e-02, -3.0836e-02, 3.8604e-02, ..., 3.5473e-02,
-4.7046e-03, -2.9440e-02],
[ 2.4811e-02, 1.2679e-02, 1.0070e-02, ..., -8.3476e-03,
1.7960e-02, -1.7406e-02],
...,
[-1.3021e-02, 2.9023e-02, -6.1800e-02, ..., -5.2802e-02,
-4.7817e-02, -2.2377e-02],
[-3.8513e-03, -1.0603e-02, -3.9712e-02, ..., 5.1941e-03,
8.2868e-03, -8.3469e-03],
[ 3.8993e-03, 3.2017e-02, -3.6292e-02, ..., -2.0210e-02,
-4.0358e-02, 1.7709e-02]],[[-1.0894e-03, 1.5720e-02, 7.0129e-03, ..., -1.2024e-02,
1.8644e-02, 1.7892e-02],
[-2.3866e-02, 9.1136e-03, 3.5243e-02, ..., -1.6756e-02,
1.4441e-03, 4.7943e-02],
[-2.0514e-03, 4.3022e-02, 2.6358e-02, ..., -2.3662e-02,
-7.8241e-04, 1.0167e-02],
...[[-4.6689e-02, -1.1407e-03, 1.8674e-02, ..., 1.2649e-03,
-2.9532e-02, 6.4535e-04],
[ 1.4171e-03, -1.9274e-02, -8.6811e-03, ..., 2.4428e-02,
6.9516e-03, 4.3715e-02],
[ 1.9982e-02, 1.3124e-02, 9.1508e-03, ..., 2.5405e-02,
-1.3132e-02, 4.0835e-02],
...,
[-3.4174e-03, 1.8623e-02, -1.4386e-02, ..., 1.0627e-03,
-5.1297e-04, 2.2055e-02],
[ 2.7333e-02, 2.4858e-02, -5.4305e-02, ..., -1.2139e-02,
1.7735e-03, -3.4184e-03],
[ 1.1412e-03, 1.5794e-02, -2.0699e-02, ..., -1.7846e-02,
3.7425e-02, -1.6059e-02]]],
...,
[[[-2.7389e-02, -3.8327e-02, -2.9043e-02, ..., -7.6396e-03,
-1.6519e-02, 3.9659e-02],
[ 2.8740e-03, -1.0621e-02, -9.2430e-03, ..., 2.2581e-02,
5.1526e-03, -2.0006e-02],
[ 1.3575e-02, 1.5290e-02, -1.7260e-02, ..., 6.3830e-03,
-1.9759e-02, 1.5501e-02],
...,
[ 1.6091e-02, 2.4038e-02, 2.4507e-02, ..., -4.5613e-02,
-3.6233e-02, 2.1632e-02],
[-1.1573e-02, -3.6514e-02, 4.1576e-02, ..., 1.8090e-02,
-2.3350e-02, -8.7074e-03],
[-1.5837e-02, -3.1353e-02, 1.8726e-02, ..., 9.3698e-03,
3.0781e-02, 1.0976e-02]],[[-2.7063e-02, 8.7158e-03, 2.7193e-03, ..., -1.6670e-03,
-4.3033e-03, 7.2011e-04],
[ 2.7870e-03, 1.4264e-02, -5.0581e-02, ..., 2.5463e-02,
7.6864e-03, -4.9655e-02],
[ 2.6030e-03, 2.5918e-02, 2.9615e-02, ..., 3.0676e-02,
-2.7723e-02, -7.3628e-03],
...,
[ 2.5969e-02, -1.4247e-02, 1.2516e-02, ..., 5.9602e-03,
-3.2843e-02, 3.5822e-02],
[ 1.2845e-02, -2.0035e-02, 9.9398e-04, ..., -3.1800e-02,
5.7984e-03, 2.8756e-02],
[ 2.3458e-02, 3.8193e-02, -2.3754e-03, ..., -1.3867e-02,
8.0831e-03, -3.2438e-02]],
...[[-9.9291e-03, -5.6023e-03, -1.7064e-02, ..., 8.8544e-03,
-5.8145e-03, 2.3248e-02],
[ 1.2148e-02, -1.0730e-02, -1.2682e-02, ..., 9.4389e-03,
1.2149e-02, 3.8613e-03],
[ 3.5913e-02, -5.2048e-04, -8.7133e-02, ..., -2.0969e-03,
-5.4117e-03, 5.4637e-05],
...,
[ 4.0351e-03, -1.3189e-02, 3.1229e-02, ..., 3.2340e-02,
-2.8351e-02, 1.0634e-02],
[ 2.6041e-02, -3.0633e-04, -1.2732e-02, ..., 2.9417e-02,
-7.3859e-03, 1.7207e-02],
[ 6.9960e-04, 3.8486e-03, 1.0397e-02, ..., 1.4535e-03,
-3.6449e-02, 3.4848e-02]]]], requires_grad=True))
********************************************************************************************************************************************************************************************************
name = bn1.weight
----------------------------------------------------------------------------------------------------
param_tuple = ('bn1.weight', Parameter containing:
tensor([1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
1., 1., 1., 1., 1., 1., 1., 1., 1., 1.], requires_grad=True))
********************************************************************************************************************************************************************************************************
name = bn1.bias
----------------------------------------------------------------------------------------------------
param_tuple = ('bn1.bias', Parameter containing:
tensor([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
requires_grad=True))
****************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************
name = layer1.0.conv2.weight
----------------------------------------------------------------------------------------------------
param_tuple = ('layer1.0.conv2.weight', Parameter containing:
tensor([[[[-8.6159e-02, 1.8507e-04, 5.4006e-03],
[-6.3063e-03, 3.9225e-03, -6.3141e-02],
[-7.0145e-02, -3.9266e-02, 1.9724e-03]],[[ 4.6454e-02, 2.1519e-02, 5.3696e-02],
[ 1.1086e-02, 1.6269e-01, -7.0579e-02],
[-1.1220e-01, -4.9811e-02, -7.5515e-02]],[[ 5.5275e-02, -8.2407e-02, -8.9807e-02],
[ 5.8418e-02, 4.4029e-02, 3.0584e-03],
[ 5.2371e-02, -1.5983e-02, 5.1494e-02]],...,
[[ 7.3441e-02, 4.5401e-02, -1.9175e-02],
[-6.2500e-02, -8.5905e-03, -7.4856e-02],
[-1.6170e-02, 3.7529e-02, -5.1231e-02]],[[ 7.7501e-04, -5.7506e-02, 1.8422e-01],
[ 2.4594e-02, 1.7378e-02, 4.0000e-02],
[-8.6796e-02, -6.0548e-02, 2.6795e-02]],[[ 5.3264e-02, -8.6190e-02, 4.2443e-02],
[-6.8029e-03, -1.6581e-02, 7.8568e-02],
[ 3.2037e-02, -7.3002e-02, 4.9353e-02]]]], requires_grad=True))
...,
********************************************************************************************************************************************************************************************************
name = layer4.1.bn2.bias
----------------------------------------------------------------------------------------------------
param_tuple = ('layer4.1.bn2.bias', Parameter containing:
tensor([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
...,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0.], requires_grad=True))
********************************************************************************************************************************************************************************************************
name = fc.weight
----------------------------------------------------------------------------------------------------
param_tuple = ('fc.weight', Parameter containing:
tensor([[-0.0125, 0.0437, -0.0014, ..., -0.0230, 0.0280, 0.0249],
[-0.0105, 0.0242, 0.0291, ..., 0.0153, 0.0366, -0.0236],
[-0.0315, 0.0306, -0.0216, ..., 0.0387, 0.0403, 0.0056],
...,
[-0.0068, -0.0222, -0.0027, ..., -0.0243, 0.0260, 0.0065],
[ 0.0213, 0.0167, -0.0379, ..., -0.0140, 0.0037, -0.0372],
[ 0.0180, 0.0101, -0.0341, ..., -0.0295, -0.0146, 0.0416]],
requires_grad=True))
********************************************************************************************************************************************************************************************************
name = fc.bias
----------------------------------------------------------------------------------------------------
param_tuple = ('fc.bias', Parameter containing:
tensor([ 3.5711e-02, 3.2682e-02, 7.5932e-03, -3.1623e-02, -9.6316e-03,
-2.4051e-02, -1.0393e-02, 2.3210e-02, -3.6044e-02, 2.3099e-02,
-3.5723e-02, -3.9482e-02, 4.8526e-03, -3.2688e-02, 3.7720e-03,
-2.2014e-02, -4.0935e-02, 4.0533e-02, -4.1172e-02, 3.9513e-02,
-3.0332e-02, 3.2777e-02, 1.3342e-02, 2.3394e-02, 8.2328e-03,
1.3757e-02, -1.7578e-02, -2.7165e-02, 3.8495e-03, -3.2116e-02,
7.9903e-03, 9.9640e-04, -8.3106e-03, 2.5033e-02, -3.0446e-02,
-1.8282e-02, -3.8420e-03, -8.6129e-03, -4.2712e-03, 1.7169e-02,
...,
-5.3570e-05, -3.7353e-02, -9.8633e-03, -9.1069e-03, 3.2688e-02,
2.2457e-02, 7.6379e-03, -3.6287e-02, -1.0444e-02, 2.1669e-02,
2.5270e-02, -4.3881e-02, 2.1960e-02, 2.6293e-02, -3.5049e-02,
-2.0074e-02, -9.7686e-03, -2.3766e-02, -5.0265e-03, -2.1095e-02,
2.0981e-02, -3.5132e-02, 8.6407e-03, 1.8453e-02, 2.4282e-02,
3.8392e-02, -1.7470e-02, 3.6958e-02, -3.7590e-02, -4.1951e-02,
-1.8246e-02, 9.0818e-03, 3.8774e-02, 7.3408e-03, 1.7728e-02,
3.5547e-02, -7.2857e-03, -2.7015e-02, -8.6983e-03, -2.3785e-02],
requires_grad=True))
********************************************************************************************************************************************************************************************************Process finished with exit code 0
二、model.named_parameters()和model.state_dict()差別
它們的差異主要體現(xiàn)在3方面:
- 返回值類型不同
- 存儲的模型參數(shù)的種類不同
- 返回的值的require_grad屬性不同
| named_parameters() | state_dict() |
|---|---|
| 將layer_name : layer_param的鍵值信息打包成一個元祖然后再存到list當(dāng)中 | 將layer_name : layer_param的鍵值信息存儲為dict形式 |
| 只保存可學(xué)習(xí)、可被更新的參數(shù),model.buffer()中的參數(shù)不包含在model.named_parameters()中 | 存儲的是該model中包含的所有l(wèi)ayer中的所有參數(shù) |
| require_grad屬性都是True | 存儲的模型參數(shù)tensor的require_grad屬性都是False |
為何model.parameters()迭代出來的所有參數(shù)的require_grad屬性都是True,因為它們在被創(chuàng)建時,默認(rèn)的require_grad就是True。
這也符合邏輯,即,使用nn.Parameter()創(chuàng)建的變量是模型參數(shù),本就是要參與學(xué)習(xí)和更新的
總結(jié)
以上為個人經(jīng)驗,希望能給大家一個參考,也希望大家多多支持腳本之家。
相關(guān)文章
python之線程通過信號pyqtSignal刷新ui的方法
今天小編就為大家分享一篇python之線程通過信號pyqtSignal刷新ui的方法,具有很好的參考價值,希望對大家有所幫助。一起跟隨小編過來看看吧2019-01-01
簡單了解Django ContentType內(nèi)置組件
這篇文章主要介紹了簡單了解Django ContentType內(nèi)置組件,文中通過示例代碼介紹的非常詳細(xì),對大家的學(xué)習(xí)或者工作具有一定的參考學(xué)習(xí)價值,需要的朋友可以參考下2019-07-07

