深度学习基本概念softmax、sigmoid、均方误差、交叉熵

今天药忘吃喽~ 2023-10-11 09:28 84阅读 0赞

在神经网络的最后一层总用到的激活函数为softmax和sigmoid。 在经过以上 激活函数后，线性回归选用均方误差为损失函数，逻辑回归选择交叉熵为损失函数。以下代码使用tensorflow实现。

softmax激活函数：用在多分类，一个输入只能有一个输出，输出标签独立（onehot类型标签，只有一个为1，其他都是0）），如猫狗大战一张图像，识别结果非猫即狗。

--------------------

![Yn=\\frac\{e^\{n\}\}\{\\sum\_\{i\}^\{k\} e^\{i\}\}][Yn_frac_e_n_sum_i_k_ e_i]

tensorflow 实现 tf.nn.sigmoid(logits)，自己定义实现如下

def Sigmoid_( logits ):
        date = tf.cast(logits,tf.int32)
        result = []
        with tf.Session() as sess:
            date = sess.run(date)
            for i in date:
                sig = round(1/(1+math.exp(-i)),6 )
                result.append(sig)
        return result

--------------------

sigmoid激活函数：用在多标签，即输入一张图像，可以同时多标签种类为1，如一张图像即有猫又有狗标签中含有两个1.

![Yn=\\frac\{1\}\{(1+e^\{x\})\}][Yn_frac_1_1_e_x]

tensorflow实现tf.nn.softmax(logits)，自定义实现

def Softmax_( logits ):
        date = tf.cast(logits,tf.int32)
        result = []
        with tf.Session() as sess:
            date = sess.run(date)
            sum = 0
            for i in date:
                sum += math.exp(i)
            for i in date:
                result.append( round(math.exp(i)/sum, 6))
            return result

--------------------

softmax的交叉熵损失函数与sigmoid的略有不同，因为softmax标签只有一张可能性

softmax:                    ![loss = -labels\[i\]\*log(logits)][loss _ -labels_i_log_logits]

sigmoid:                  ![loss = -labels\[i\]\*log(logits) - (1-labels\[i\])\*log(1-logits)][loss _ -labels_i_log_logits_ - _1-labels_i_log_1-logits]

整体实现代码：

# coding:utf-8
    import tensorflow as tf
    import os
    import math
    os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'
    
    
    def Sigmoid_( logits ):
        date = tf.cast(logits,tf.int32)
        result = []
        with tf.Session() as sess:
            date = sess.run(date)
            for i in date:
                sig = round(1/(1+math.exp(-i)),6 )
                result.append(sig)
        return result
    
    
    def Softmax_( logits ):
        date = tf.cast(logits,tf.int32)
        result = []
        with tf.Session() as sess:
            date = sess.run(date)
            sum = 0
            for i in date:
                sum += math.exp(i)
            for i in date:
                result.append( round(math.exp(i)/sum, 6))
            return result
    
    
    def get_loss( res1 ):
        print("softmax loss")
        labels = [0, 1, 0]
        loss_sum = 0
        for i in range(len(labels)):
            loss_ = -labels[i]*tf.log(res1[i])
            loss_sum =loss_sum+loss_
        return  loss_sum
    
    
    def get_sigmoid_loss( res1 ):
        print("sigmoid loss")
        labels = [0, 1, 0]
        loss_sum = []
        for i in range(len(labels)):
            loss_ = -labels[i]*tf.log(res1[i]) - (1-labels[i])*tf.log(1-res1[i])
            loss_sum.append(loss_)
        return  loss_sum
    
    print('=====================sigmod test===============================')
    
    logits = tf.constant([2,5,1],dtype=tf.float32)
    labels = [0,1,0]
    label = tf.convert_to_tensor(labels)
    label = tf.cast(label,tf.float32)
    
    res_sigmoid = tf.nn.sigmoid(logits)
    
    ls = get_sigmoid_loss(res_sigmoid)
    res2 = Sigmoid_(logits)
    res3 = tf.nn.sigmoid_cross_entropy_with_logits(logits=logits,labels=label)
    
    with tf.Session() as sess:
        res1,res3 = sess.run([res_sigmoid,res3])
        los = sess.run(ls)
        print("tf计算结果：",   res1)
        print("手写计算结果：",  res2)
        print("tf计算结果：",   res3)
        print("手写计算结果：",  los)
    
    print('====================softmax test================================')
    res1 = tf.nn.softmax(logits)
    ls = get_loss(res1)
    
    res2 = Softmax_(logits)
    res3 = tf.nn.softmax_cross_entropy_with_logits_v2(logits=logits,labels=labels)
    with tf.Session() as sess:
        res1,res3 = sess.run([res1,res3])
        los = sess.run(ls)
        print("tf计算结果：",   res1)
        print("手写计算结果：", res2)
        print("tf计算结果：",   res3)
        print("手写计算结果：",  los)
    
    print('====================================================')

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L1RZdGFuZ3lhbg_size_16_color_FFFFFF_t_70][]

[Yn_frac_e_n_sum_i_k_ e_i]: https://private.codecogs.com/gif.latex?Yn%3D%5Cfrac%7Be%5E%7Bn%7D%7D%7B%5Csum_%7Bi%7D%5E%7Bk%7D%20e%5E%7Bi%7D%7D
[Yn_frac_1_1_e_x]: https://private.codecogs.com/gif.latex?Yn%3D%5Cfrac%7B1%7D%7B%281&plus;e%5E%7Bx%7D%29%7D
[loss _ -labels_i_log_logits]: https://private.codecogs.com/gif.latex?loss%20%3D%20-labels%5Bi%5D*log%28logits%29
[loss _ -labels_i_log_logits_ - _1-labels_i_log_1-logits]: https://private.codecogs.com/gif.latex?loss%20%3D%20-labels%5Bi%5D*log%28logits%29%20-%20%281-labels%5Bi%5D%29*log%281-logits%29
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L1RZdGFuZ3lhbg_size_16_color_FFFFFF_t_70]: https://img-blog.csdnimg.cn/20190922140933742.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L1RZdGFuZ3lhbg==,size_16,color_FFFFFF,t_70