devbox
weixin_40725706
这个屌丝很懒，什么也没留下！
关注作者
热门标签
热门文章
当前位置:   article > 正文

循环神经网络实现文本情感分类之使用LSTM完成文本情感分类_lstm情感分类

作者：weixin_40725706 | 2024-04-03 07:24:18

赞

踩

lstm情感分类

循环神经网络实现文本情感分类之使用LSTM完成文本情感分类

1. 使用LSTM完成文本情感分类

在前面，使用了word embedding去实现了toy级别的文本情感分类，那么现在在这个模型中添加上LSTM层，观察分类效果。

为了达到更好的效果，对之前的模型做如下修改

  1. MAX_LEN = 200

  2. 构建dataset的过程，把数据转化为2分类的问题，pos为1，neg为0，否则25000个样本完成10个类别的划分数据量是不够的

  3. 在实例化LSTM的时候，使用dropout=0.5，在model.eval()的过程中，dropout自动会为0

1.1 修改模型

  1. import torch
  2. import pickle
  3. import torch.nn as nn
  4. import torch.nn.functional as F
  5.  
  6. ws = pickle.load(open('./model/ws.pkl', 'rb'))
  7. device = torch.device('cuda:0' if torch.cuda.is_available() else 'cpu')
  8.  
  9.  
  10. class IMDBLstmModel(nn.Module):
  11.     def __init__(self):
  12.         super().__init__()
  13.         self.embedding_dim = 200
  14.         self.hidden_size = 64
  15.         self.num_layer = 2
  16.         self.bidirectional = True
  17.         self.bi_num = 2 if self.bidirectional else 1
  18.         self.dropout = 0.5
  19.         #  以上部分为超参数，可以自行修改
  20.  
  21.         self.embedding = nn.Embedding(len(ws), self.embedding_dim, padding_idx=ws.PAD)  # [N, 300]
  22.         self.lstm = nn.LSTM(self.embedding_dim, self.hidden_size, self.num_layer, bidirectional=self.bidirectional,
  23.                             dropout=self.dropout)
  24.  
  25.         #  使用两个全连接层，中间使用relu激活函数
  26.         self.fc = nn.Linear(self.hidden_size * self.bi_num, 20)
  27.         self.fc2 = nn.Linear(20, 2)
  28.  
  29.     def forward(self, x):
  30.         x = self.embedding(x)
  31.         x = x.permute(1, 0, 2)  # 进行轴交换
  32.         h_0, c_0 = self.init_hidden_state(x.size(1))
  33.         _, (h_n, c_0) = self.lstm(x, (h_0, c_0))
  34.  
  35.         #  只要最后一个lstm单元处理的结果，这里去掉了hidden_state
  36.         out = torch.cat([h_n[-2, :, :], h_n[-1, :, :]], dim=-1)
  37.         out = self.fc(out)
  38.         out = F.relu(out)
  39.         out = self.fc2(out)
  40.         return F.log_softmax(out, dim=-1)
  41.  
  42.     def init_hidden_state(self, batch_size):
  43.         h_0 = torch.rand(self.num_layer * self.bi_num, batch_size, self.hidden_size).to(device)
  44.         c_0 = torch.rand(self.num_layer * self.bi_num, batch_size, self.hidden_size).to(device)
  45.         return h_0, c_0

2.2 完成训练和测试代码

为了提高程序的运行速度，可以考虑把模型放在gup上运行，那么此时需要处理一下几点：

  1. device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

  2. model.to(device)

  3. 除了上述修改外，涉及计算的所有tensor都需要转化为CUDA的tensor

    1. 初始化的h_0,c_0

    2. 训练集和测试集的input,traget

  4. 在最后可以通过tensor.cpu()转化为torch的普通tensor

  1. from torch import optim
  2.  
  3. train_batch_size = 64
  4. test_batch_size = 5000
  5. # imdb_model = IMDBLstmModel(MAX_LEN)  # 基础model
  6. imdb_model = IMDBLstmModel().to(device)  # 在GPU上运行，提高运行速度
  7. # imdb_model.load_state_dict(torch.load("model/
  8. optimizer = optim.Adam(imdb_model.parameters())
  9. criterion = nn.CrossEntropyLoss()
  10.  
  11.  
  12. def train(epoch):
  13.     mode = True
  14.     imdb_model.train(mode)
  15.     train_dataloader = get_dataloader(mode, train_batch_size)
  16.     for idx, (target, input, input_length) in enumerate(train_dataloader):
  17.         target = target.to(device)
  18.         input = input.to(device)
  19.         optimizer.zero_grad()
  20.         output = imdb_model(input)
  21.         loss = F.nll_loss(output, target)  # target需要是[0,9],不能是[1-10]
  22.         loss.backward()
  23.         optimizer.step()
  24.         if idx % 10 == 0:
  25.             pred = torch.max(output, dim=-1, keepdim=False)[-1]
  26.             acc = pred.eq(target.data).cpu().numpy().mean() * 100.  # 使用eq判断是否一致
  27.             print('Train Epoch: {} [{}/{} ({:.0f}%)]\tLoss: {:.6f}\t ACC: {:.6f}'.format(epoch, idx * len(input),
  28.                                                                                          len(train_dataloader.dataset),
  29.                                                                                          100. * idx / len(
  30.                                                                                              train_dataloader),
  31.                                                                                          loss.item(), acc))
  32.  
  33.             torch.save(imdb_model.state_dict(), "model/mnist_net.pkl")
  34.             torch.save(optimizer.state_dict(), 'model/mnist_optimizer.pkl')
  35.  
  36.  
  37. def test():
  38.     mode = False
  39.     imdb_model.eval()
  40.     test_dataloader = get_dataloader(mode, test_batch_size)
  41.     with torch.no_grad():
  42.         for idx, (target, input, input_lenght) in enumerate(test_dataloader):
  43.             target = target.to(device)
  44.             input = input.to(device)
  45.             output = imdb_model(input)
  46.             test_loss = F.nll_loss(output, target, reduction="mean")
  47.             pred = torch.max(output, dim=-1, keepdim=False)[-1]
  48.             correct = pred.eq(target.data).sum()
  49.             acc = 100. * pred.eq(target.data).cpu().numpy().mean()
  50.             print('idx: {} Test set: Avg. loss: {:.4f}, Accuracy: {}/{} ({:.2f}%)\n'.format(idx, test_loss, correct,
  51.                                                                                             target.size(0), acc))
  52.  
  53.  
  54. if __name__ == "__main__":
  55.     test()
  56.     for i in range(10):
  57.         train(i)
  58.         test()

2.3 模型训练的最终输出

  1. ...
  2. Train Epoch: 9 [20480/25000 (82%)]	Loss: 0.017165	 ACC: 100.000000
  3. Train Epoch: 9 [21120/25000 (84%)]	Loss: 0.021572	 ACC: 98.437500
  4. Train Epoch: 9 [21760/25000 (87%)]	Loss: 0.058546	 ACC: 98.437500
  5. Train Epoch: 9 [22400/25000 (90%)]	Loss: 0.045248	 ACC: 98.437500
  6. Train Epoch: 9 [23040/25000 (92%)]	Loss: 0.027622	 ACC: 98.437500
  7. Train Epoch: 9 [23680/25000 (95%)]	Loss: 0.097722	 ACC: 95.312500
  8. Train Epoch: 9 [24320/25000 (97%)]	Loss: 0.026713	 ACC: 98.437500
  9. Train Epoch: 9 [15600/25000 (100%)]	Loss: 0.006082	 ACC: 100.000000
  10. idx: 0 Test set: Avg. loss: 0.8794, Accuracy: 4053/5000 (81.06%)
  11. idx: 1 Test set: Avg. loss: 0.8791, Accuracy: 4018/5000 (80.36%)
  12. idx: 2 Test set: Avg. loss: 0.8250, Accuracy: 4087/5000 (81.74%)
  13. idx: 3 Test set: Avg. loss: 0.8380, Accuracy: 4074/5000 (81.48%)
  14. idx: 4 Test set: Avg. loss: 0.8696, Accuracy: 4027/5000 (80.54%)

可以看到模型的测试准确率稳定在81%左右。

大家可以把上述代码改为GRU，或者多层LSTM继续尝试，观察效果

完整代码：

目录结构：

main.py

  1. # 由于pickle特殊性，需要在此导入Word2Sequence
  2. from word_squence import Word2Sequence
  3. import pickle
  4. import os
  5. from dataset import tokenlize
  6. from tqdm import tqdm  # 显示当前迭代进度
  7.  
  8. TRAIN_PATH = r"../data/aclImdb/train"
  9.  
  10. if __name__ == '__main__':
  11.     ws = Word2Sequence()
  12.     temp_data_path = [os.path.join(TRAIN_PATH, 'pos'), os.path.join(TRAIN_PATH, 'neg')]
  13.     for data_path in temp_data_path:
  14.         # 获取每一个文件的路径
  15.         file_paths = [os.path.join(data_path, file_name) for file_name in os.listdir(data_path)]
  16.         for file_path in tqdm(file_paths):
  17.             sentence = tokenlize(open(file_path, errors='ignore').read())
  18.             ws.fit(sentence)
  19.     ws.build_vocab(max=10, max_features=10000)
  20.     pickle.dump(ws, open('./model/ws.pkl', 'wb'))
  21.     print(len(ws.dict))

model.py

  1. """
  2. 定义模型
  3. 模型优化方法：
  4. # 为使得结果更好 添加一个新的全连接层，作为输出，激活函数处理
  5. # 把双向LSTM的output传给一个单向LSTM再进行处理
  7. lib.max_len = 200
  8. lib.embedding_dim = 100  # 用长度为100的向量表示一个词
  9. lib.hidden_size = 128  # 每个隐藏层中LSTM单元个数
  10. lib.num_layer = 2  # 隐藏层数量
  11. lib.bidirectional = True  # 是否双向LSTM
  12. lib.dropout = 0.3  # 在训练时以一定的概率使神经元失活，实际上就是让对应神经元的输出为0
  13. lib.device = torch.device('cuda:0' if torch.cuda.is_available() else 'cpu')
  14. """
  15. import torch.nn as nn
  16. from lib import ws
  17. import torch.nn.functional as F
  18. from torch.optim import Adam
  19. from dataset import get_dataloader
  20. from tqdm import tqdm
  21. import torch
  22. import numpy as np
  23. import lib
  24. import os
  25.  
  26.  
  27. class Mymodel(nn.Module):
  28.     def __init__(self):
  29.         super().__init__()
  30.         # nn.Embedding(num_embeddings - 词嵌入字典大小即一个字典里要有多少个词，embedding_dim - 每个词嵌入向量的大小。)
  31.         self.embedding = nn.Embedding(len(ws), 100)
  32.         # 加入LSTM
  33.         self.lstm = nn.LSTM(input_size=lib.embedding_dim, hidden_size=lib.hidden_size, num_layers=lib.num_layer,
  34.                             batch_first=True, bidirectional=lib.bidirectional, dropout=lib.dropout)
  35.         self.fc = nn.Linear(lib.hidden_size * 2, 2)
  36.  
  37.     def forward(self, input):
  38.         """
  39.         :param input: 形状[batch_size, max_len]
  40.         :return:
  41.         """
  42.         x = self.embedding(input)  # 进行embedding，形状[batch_size, max_len, 100]
  43.  
  44.         # x [batch_size, max_len, num_direction*hidden_size]
  45.         # h_n[num_direction * num_layer, batch_size, hidden_size]
  46.         x, (h_n, c_n) = self.lstm(x)
  47.         # 获取两个方向最后一次的output(正向最后一个和反向第一个)进行concat
  48.         # output = x[:,-1,:hidden_size]   前向，等同下方
  49.         output_fw = h_n[-2, :, :]  # 正向最后一次输出
  50.         # output = x[:,0,hidden_size:]   反向，等同下方
  51.         output_bw = h_n[-1, :, :]  # 反向最后一次输出
  52.         #  只要最后一个lstm单元处理的结果，这里去掉了hidden state
  53.         output = torch.cat([output_fw, output_bw], dim=-1)  # [batch_size, hidden_size*num_direction]
  54.  
  55.         out = self.fc(output)
  56.  
  57.         return F.log_softmax(out, dim=-1)
  58.  
  59.  
  60. model = Mymodel()
  61. optimizer = Adam(model.parameters(), lr=0.01)
  62. if os.path.exists('./model/model.pkl'):
  63.     model.load_state_dict(torch.load('./model/model.pkl'))
  64.     optimizer.load_state_dict(torch.load('./model/optimizer.pkl'))
  65.  
  66.  
  67. # 训练
  68. def train(epoch):
  69.     for idx, (input, target) in enumerate(get_dataloader(train=True)):
  70.         output = model(input)
  71.         optimizer.zero_grad()
  72.         loss = F.nll_loss(output, target)
  73.         loss.backward()
  74.         optimizer.step()
  75.         print(loss.item())
  76.         print('当前第%d轮,idx为%d 损失为:%lf, ' % (epoch, idx, loss.item()))
  77.  
  78.         # 保存模型
  79.         if idx % 100 == 0:
  80.             torch.save(model.state_dict(), './model/model.pkl')
  81.             torch.save(optimizer.state_dict(), './model/optimizer.pkl')
  82.  
  83.  
  84. # 评估
  85. def test():
  86.     acc_list = []
  87.     loss_list = []
  88.     # 开启模型评估模式
  89.     model.eval()
  90.     # 获取测试集数据
  91.     test_dataloader = get_dataloader(train=False)
  92.     # tqdm(total = 总数,ascii = #,desc=描述)
  93.     for idx, (input, target) in tqdm(enumerate(test_dataloader), total=len(test_dataloader), ascii=True, desc='评估：'):
  94.         with torch.no_grad():
  95.             output = model(input)
  96.             # 计算当前损失
  97.             cur_loss = F.nll_loss(output, target)
  98.             loss_list.append(cur_loss)
  99.             pred = output.max(dim=-1)[-1]
  100.             # 计算当前准确率
  101.             cur_acc = pred.eq(target).float().mean()
  102.             acc_list.append(cur_acc)
  103.     print('准确率为:%lf, 损失为:%lf' % (np.mean(acc_list), np.mean(loss_list)))
  104.  
  105.  
  106. if __name__ == '__main__':
  107.     for i in tqdm(range(10)):
  108.         train(i)
  109.     test()

dataset.py:

  1. import torch
  2. from torch.utils.data import Dataset, DataLoader
  3. import os
  4. import re
  5.  
  6. """
  7. 完成数据集准备
  8. """
  9. TRAIN_PATH = r"..\data\aclImdb\train"
  10. TEST_PATH = r"..\data\aclImdb\test"
  11.  
  12.  
  13. # 分词
  14. def tokenlize(content):
  15.     content = re.sub(r"<.*?>", " ", content)
  16.     filters = ['!', '"', '#', '$', '%', '&', '\(', '\)', '\*', '\+', ',', '-', '\.', '/', ':', ';', '<', '=', '>', '\?',
  17.                '@', '\[', '\\', '\]', '^', '_', '`', '\{', '\|', '\}', '~', '\t', '\n', '\x97', '\x96', '”', '“', ]
  18.     content = re.sub("|".join(filters), " ", content)
  19.     tokens = [i.strip().lower() for i in content.split()]
  20.     return tokens
  21.  
  22.  
  23. class ImbdDateset(Dataset):
  24.     def __init__(self, train=True):
  25.         self.train_data_path = TRAIN_PATH
  26.         self.test_data_path = TEST_PATH
  27.         # 通过train和data_path控制读取train或者test数据集
  28.         data_path = self.train_data_path if train else self.test_data_path
  29.         # 把所有文件名放入列表
  30.         # temp_data_path = [data_path + '/pos', data_path + '/neg']
  31.         temp_data_path = [os.path.join(data_path, 'pos'), os.path.join(data_path, 'neg')]
  32.         self.total_file_path = []  # 所有pos,neg评论文件的path
  33.         # 获取每个文件名字，并拼接路径
  34.         for path in temp_data_path:
  35.             file_name_list = os.listdir(path)
  36.             file_path_list = [os.path.join(path, i) for i in file_name_list if i.endswith('.txt')]
  37.             self.total_file_path.extend(file_path_list)
  38.  
  39.     def __getitem__(self, index):
  40.         # 获取index的path
  41.         file_path = self.total_file_path[index]
  42.         # 获取label
  43.         label_str = file_path.split('\\')[-2]
  44.         label = 0 if label_str == 'neg' else 1
  45.         # 获取content
  46.         tokens = tokenlize(open(file_path, errors='ignore').read())
  47.         return tokens, label
  48.  
  49.     def __len__(self):
  50.         return len(self.total_file_path)
  51.  
  52.  
  53. def get_dataloader(train=True):
  54.     imdb_dataset = ImbdDateset(train)
  55.     data_loader = DataLoader(imdb_dataset, shuffle=True, batch_size=128, collate_fn=collate_fn)
  56.     return data_loader
  57.  
  58.  
  59. # 重新定义collate_fn
  60. def collate_fn(batch):
  61.     """
  62.     :param batch: (一个__getitem__[tokens, label], 一个__getitem__[tokens, label],..., batch_size个)
  63.     :return:
  64.     """
  65.     content, label = list(zip(*batch))
  66.     from lib import ws, max_len
  67.     content = [ws.transform(i, max_len=max_len) for i in content]
  68.     content = torch.LongTensor(content)
  69.     label = torch.LongTensor(label)
  70.     return content, label
  71.  
  72.  
  73. if __name__ == '__main__':
  74.     for idx, (input, target) in enumerate(get_dataloader()):
  75.         print(idx)
  76.         print(input)
  77.         print(target)
  78.         break

word_squence.py

  1. import numpy as np
  2.  
  3. """
  4. 构建词典,实现方法把句子转换为序列,和其翻转
  5. """
  6.  
  7.  
  8. class Word2Sequence(object):
  9.     # 2个特殊类属性，标记特殊字符和填充标记
  10.     UNK_TAG = 'UNK'
  11.     PAD_TAG = 'PAD'
  12.  
  13.     UNK = 0
  14.     PAD = 1
  15.  
  16.     def __init__(self):
  17.         self.dict = {
  18.             # 保存词语和对应的数字
  19.             self.UNK_TAG: self.UNK,
  20.             self.PAD_TAG: self.PAD
  21.         }
  22.         self.count = {}  # 统计词频
  23.  
  24.     def fit(self, sentence):
  25.         """
  26.         把单个句子保存到dict中
  27.         :param sentence: [word1, word2 , ... , ]
  28.         :return:
  29.         """
  30.         for word in sentence:
  31.             # 对word出现的频率进行统计，当word不在sentence时，返回值是0，当word在sentence中时，返回+1，以此进行累计计数
  32.             self.count[word] = self.count.get(word, 0) + 1
  33.  
  34.     def build_vocab(self, min=5, max=None, max_features=None):
  35.         """
  36.         生成词典
  37.         :param min:最小词频数
  38.         :param max:最大词频数
  39.         :param max_feature:一共保留多少词语
  40.         :return:
  41.         """
  42.         # 删除count < min 的词语,即保留count > min 的词语
  43.         if min is not None:
  44.             self.count = {word: value for word, value in self.count.items() if value > min}
  45.         # 删除count > min 的词语,即保留count < max 的词语
  46.         if max is not None:
  47.             self.count = {word: value for word, value in self.count.items() if value < max}
  48.         # 限制保留的词语数
  49.         if max_features is not None:
  50.             # sorted 返回一个列表[(key1, value1), (key2, value2),...]，True为升序
  51.             temp = sorted(self.count.items(), key=lambda x: x[-1], reverse=True)[:max_features]
  52.             self.count = dict(temp)
  53.             for word in self.count:
  54.                 self.dict[word] = len(self.dict)
  55.  
  56.         # 得到一个翻转的dict字典
  57.         # zip方法要比{value: word for word, value in self.dict.items()}快
  58.         self.inverse_dict = dict(zip(self.dict.values(), self.dict.keys()))
  59.  
  60.     def transform(self, sentence, max_len=None):
  61.         """
  62.         把句子转换为序列
  63.         :param sentence: [word1, word2...]
  64.         :param max_len: 对句子进行填充或者裁剪
  65.         :return:
  66.         """
  67.         if max_len is not None:
  68.             # 句子长度小于最大长度，进行填充
  69.             if max_len > len(sentence):
  70.                 sentence = sentence + [self.PAD_TAG] * (max_len - len(sentence))
  71.             # 句子长度大于最大长度，进行裁剪
  72.             if max_len < len(sentence):
  73.                 sentence = sentence[:max_len]
  74.         # for word in sentence:
  75.         #     self.dict.get(word, self.UNK)
  76.         # 字典的get(key, default=None) 如果指定键不存在，则返回默认值None。
  77.         return [self.dict.get(word, self.UNK) for word in sentence]
  78.  
  79.     def inverse_transform(self, indices):
  80.         """
  81.         把序列转换为句子
  82.         :param indices: [1, 2, 3, ...]
  83.         :return:
  84.         """
  85.         return [self.inverse_dict.get(idx) for idx in indices]
  86.  
  87.     def __len__(self):
  88.         return len(self.dict)
  89.  
  90.  
  91. if __name__ == '__main__':
  92.     ws = Word2Sequence()
  93.     ws.fit(["我", "是", "我"])
  94.     ws.fit(["我", "是", "谁"])
  95.     ws.build_vocab(min=1, max_features=5)
  96.     print(ws.dict)
  97.     ret = ws.transform(['我', '爱', '北京'], max_len=10)
  98.     print(ret)
  99.     print(ws.inverse_transform(ret))

lib.py

  1. import pickle
  2. import torch
  3.  
  4. ws = pickle.load(open('./model/ws.pkl', 'rb'))
  5.  
  6. max_len = 200
  7. embedding_dim = 100  # 用长度为100的向量表示一个词
  8. hidden_size = 128  # 每个隐藏层中LSTM单元个数
  9. num_layer = 2  # 隐藏层数量
  10. bidirectional = True  # 是否双向LSTM
  11. dropout = 0.3  # 在训练时以一定的概率使神经元失活，实际上就是让对应神经元的输出为0
  12.  
  13. device = torch.device('cuda:0' if torch.cuda.is_available() else 'cpu')

 

声明：本文内容由网友自发贡献，不代表【wpsshop博客】立场，版权归原作者所有，本站不承担相应法律责任。如您发现有侵权的内容，请联系我们。转载请注明出处：https://www.wpsshop.cn/w/weixin_40725706/article/detail/354724?site
推荐阅读
相关标签

Copyright © 2003-2013 www.wpsshop.cn 版权所有，并保留所有权利。

  

闽ICP备14008679号