Graph Attention Network 图注意力网络 (一) 训练运行与代码概览

秒速五厘米 2022-02-18 09:37 973阅读 0赞

**目的**：运行并粗略看懂Graph attention network的pytorch代码。

**代码地址**：[https://github.com/Diego999/pyGAT][https_github.com_Diego999_pyGAT]

**论文地址**：This is a pytorch implementation of the Graph Attention Network (GAT) model presented by Veličković et. al (2017, [https://arxiv.org/abs/1710.10903][https_arxiv.org_abs_1710.10903]).  ICLR 2018

**目录**

一、配置环境与运行

1.1 安装并配置pytorch

1.2 配置requirements.txt

1.3 运行结果

二、代码

2.1 结构

2.2 训练代码

训练前过程

训练

测试

逐个epoch训练

2.3 数据加载与模型结构定义

--------------------

# 一、配置环境与运行 #

Graph Attention Network为了避免与GAN弄混，因此缩写为GAT。

## 1.1 安装并配置pytorch ##

[CentOS 6.3安装anaconda并配置pytorch与cuda ][CentOS 6.3_anaconda_pytorch_cuda]

配置完成后，source activate torch

## 1.2 配置requirements.txt ##

需要的环境：

numpy==1.15.1
    scipy==1.1.0
    torch==0.4.1.post2

python train.py 时warning：

train.py:96: UserWarning: invalid index of a 0-dim tensor. This will be an error in PyTorch 0.5. Use tensor.item() to convert a 0-dim tensor to a Python number  
  'loss\_train: \{:.4f\}'.format(loss\_train.data\[0\]),

因为torch版本问题，可能会有报警，但是不影响训练与运行。

pip install -r requirements.txt

报错：

Could not find a version that satisfies the requirement torch==0.4.1.post2 (from -r requirements.txt (line 3)) (from versions: 0.1.2, 0.1.2.post1, 0.3.1, 0.4.0, 0.4.1, 1.0.0, 1.0.1, 1.0.1.post2)  
No matching distribution found for torch==0.4.1.post2 (from -r requirements.txt (line 3))

解决，先更新pip，pip install --upgrade pip。再运行pip install -r requirements.txt

依然无法安装。将其他两项安装好，此项影响不大。可以正常运行，先不管。

## 1.3 运行结果 ##

python train.py

Epoch: 0002 loss_train: 1.9417 acc_train: 0.2143 loss_val: 1.9261 acc_val: 0.4600 time: 1.4492s
    Epoch: 0003 loss_train: 1.9287 acc_train: 0.2786 loss_val: 1.9161 acc_val: 0.4867 time: 1.4498s
    。。。
    Epoch: 0743 loss_train: 0.6635 acc_train: 0.7714 loss_val: 0.6587 acc_val: 0.8167 time: 1.4533s
    Epoch: 0744 loss_train: 0.5009 acc_train: 0.8429 loss_val: 0.6582 acc_val: 0.8200 time: 1.4527s
    Optimization Finished!
    Total time elapsed: 1085.0045s
    Loading 643th epoch
    Test set results: loss= 0.6640 accuracy= 0.8440

对比于GCN，GAT的运行时间较长且迭代实验较长。

# 二、代码 #

## 2.1 结构 ##

GCN与GAT代码结构非常类似

*  train.py模型训练代码
 *  models.py定义模型GAT与sparseGAT
 *  layers.py定义GAT layer与sparse GAT layer
 *  utils.py定义数据的加载，预处理，与normalize adj矩阵
 *  visualize\_graph.py ：Produces Graphviz representation of PyTorch autograd grap

## 2.2 训练代码 ##

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl8zNjQ3NDgwOQ_size_16_color_FFFFFF_t_70][]

GAT的训练代码与GCN的训练代码非常类似：[Graph Convolution Network图卷积网络（一）训练运行与代码概览 ][Graph Convolution Network_]

### 训练前过程 ###

加载参数，生成随机种子，加载数据

![20190422103600323.png][]

args = parser.parse_args()
    args.cuda = not args.no_cuda and torch.cuda.is_available()
    
    random.seed(args.seed)
    np.random.seed(args.seed)
    torch.manual_seed(args.seed)
    if args.cuda:
        torch.cuda.manual_seed(args.seed)
    
    # Load data
    adj, features, labels, idx_train, idx_val, idx_test = load_data()

定义模型与优化器

# Model and optimizer
    if args.sparse:
        model = SpGAT(nfeat=features.shape[1], 
                    nhid=args.hidden, 
                    nclass=int(labels.max()) + 1, 
                    dropout=args.dropout, 
                    nheads=args.nb_heads, 
                    alpha=args.alpha)
    else:
        model = GAT(nfeat=features.shape[1], 
                    nhid=args.hidden, 
                    nclass=int(labels.max()) + 1, 
                    dropout=args.dropout, 
                    nheads=args.nb_heads, 
                    alpha=args.alpha)
    optimizer = optim.Adam(model.parameters(), 
                           lr=args.lr, 
                           weight_decay=args.weight_decay)

### 训练 ###

定义每个epoch的训练

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl8zNjQ3NDgwOQ_size_16_color_FFFFFF_t_70 1][]

所有pytorch的训练基本为这个流程

def train(epoch):
        t = time.time()
        model.train()
        optimizer.zero_grad()
        output = model(features, adj)
        loss_train = F.nll_loss(output[idx_train], labels[idx_train])
        acc_train = accuracy(output[idx_train], labels[idx_train])
        loss_train.backward()
        optimizer.step()
    
        if not args.fastmode:
            # Evaluate validation set performance separately,
            # deactivates dropout during validation run.
            model.eval()
            output = model(features, adj)
    
        loss_val = F.nll_loss(output[idx_val], labels[idx_val])
        acc_val = accuracy(output[idx_val], labels[idx_val])
        print('Epoch: {:04d}'.format(epoch+1),
              'loss_train: {:.4f}'.format(loss_train.data[0]),
              'acc_train: {:.4f}'.format(acc_train.data[0]),
              'loss_val: {:.4f}'.format(loss_val.data[0]),
              'acc_val: {:.4f}'.format(acc_val.data[0]),
              'time: {:.4f}s'.format(time.time() - t))
    
        return loss_val.data[0]

### 测试 ###

![20190422104604825.png][]

def compute_test():
        model.eval()
        output = model(features, adj)
        loss_test = F.nll_loss(output[idx_test], labels[idx_test])
        acc_test = accuracy(output[idx_test], labels[idx_test])
        print("Test set results:",
              "loss= {:.4f}".format(loss_test.data[0]),
              "accuracy= {:.4f}".format(acc_test.data[0]))

### 逐个epoch训练 ###

![20190422104648716.png][]

for epoch in range(args.epochs):
        loss_values.append(train(epoch))
    
        torch.save(model.state_dict(), '{}.pkl'.format(epoch))
        if loss_values[-1] < best:
            best = loss_values[-1]
            best_epoch = epoch
            bad_counter = 0
        else:
            bad_counter += 1
    
        if bad_counter == args.patience:
            break
    
        files = glob.glob('*.pkl')
        for file in files:
            epoch_nb = int(file.split('.')[0])
            if epoch_nb < best_epoch:
                os.remove(file)

这里可以注意一个结构，训练停止的标志。就是如果新的loss比上一次大，则加一个bad\_counter,然后如果bad\_counter达到了patience，则停止训练。

## 2.3 数据加载与模型结构定义 ##

这部分较为重要，在后面详细描述。

*  models.py定义模型GAT与sparseGAT
 *  layers.py定义GAT layer与sparse GAT layer
 *  utils.py定义数据的加载，预处理，与normalize adj矩阵
 *  visualize\_graph.py ：Produces Graphviz representation of PyTorch autograd grap
 *  模型定义见[Graph Attention Network (二) 模型定义 ][Graph Attention Network _ _]

[https_github.com_Diego999_pyGAT]: https://github.com/Diego999/pyGAT
[https_arxiv.org_abs_1710.10903]: https://arxiv.org/abs/1710.10903
[CentOS 6.3_anaconda_pytorch_cuda]: https://blog.csdn.net/weixin_36474809/article/details/88799164
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl8zNjQ3NDgwOQ_size_16_color_FFFFFF_t_70]: /images/20220218/3acafddb5b93426cb1d8706c7e7ab573.png
[Graph Convolution Network_]: https://blog.csdn.net/weixin_36474809/article/details/89373123
[20190422103600323.png]: /images/20220218/7d40c67e49e943d9aec40da6b8eb58d3.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl8zNjQ3NDgwOQ_size_16_color_FFFFFF_t_70 1]: /images/20220218/4562adf8cf884fb5bbb18fc933eb5da4.png
[20190422104604825.png]: /images/20220218/e5cf32812b0a4697bfcc61924b7f950d.png
[20190422104648716.png]: /images/20220218/a4b817eef9a346e195be85e895b99ae9.png
[Graph Attention Network _ _]: https://blog.csdn.net/weixin_36474809/article/details/89447533