张量流示例，但中间层

Question

我正在尝试让这个代码工作。它可能看起来不像，但它主要来自TensorFlow mnist示例。我试图得到三层，虽然，我已经改变了输入和输出的大小。输入大小为12，中间大小为6，输出大小为2。这就是我运行时发生的事情。它不会抛出错误，但是当我运行test选项时，我总是得到50%。当我重新开始训练时，它会跑，我确信重量在变化。有保存模型和重量的代码，所以我很有信心它不会在每次我重新启动它的时候抹去我的重量。self.d\u y\u out背后的想法是有一些东西可以让我运行模型，并只获得一个图像的输出。我认为问题就在“问题？？”的评论附近。你知道吗

        self.d_keep = tf.placeholder(tf.float32)
        self.d_W_2 = tf.Variable(tf.random_normal([mid_num, output_num], stddev=0.0001))
        self.d_b_2 = tf.Variable(tf.random_normal([output_num], stddev=0.5))

        self.d_x = tf.placeholder(tf.float32, [None, input_num])
        self.d_W_1 = tf.Variable(tf.random_normal([input_num, mid_num], stddev=0.0001))  # 0.0004
        self.d_b_1 = tf.Variable(tf.zeros([mid_num]))

        self.d_y_ = tf.placeholder(tf.float32, [None, output_num])

        self.d_x_drop = tf.nn.dropout(self.d_x, self.d_keep)

        self.d_y_logits_1 = tf.matmul(self.d_x_drop, self.d_W_1) + self.d_b_1
        self.d_y_mid = tf.nn.relu(self.d_y_logits_1) 
        self.d_y_mid_drop = tf.nn.dropout(self.d_y_mid, self.d_keep)

        self.d_y_logits_2 = tf.matmul(self.d_y_mid_drop, self.d_W_2) + self.d_b_2

        self.d_y_softmax = tf.nn.softmax_cross_entropy_with_logits(logits=self.d_y_logits_2, labels=self.d_y_)

        self.d_cross_entropy = tf.reduce_mean(self.d_y_softmax) ## PROBLEM??

        self.d_train_step = tf.train.GradientDescentOptimizer(0.001).minimize(self.d_cross_entropy)  # 0.0001

        # train_step = tf.train.AdamOptimizer(1e-4).minimize(cross_entropy) #0.5

        #self.d_y_out = tf.argmax(self.d_y, 1)  ## for prediction
        self.d_y_out = tf.argmax(self.d_y_logits_2, 1, name="d_y_out")

    if self.train :

        for i in range(self.start_train,self.cursor_tot): #1000
            batch_xs, batch_ys = self.get_nn_next_train(self.batchsize)
            self.sess.run(self.d_train_step, feed_dict={self.d_x: batch_xs, self.d_y_: batch_ys, self.d_keep: 0.5})
            if True: #mid_num > 0:
                cost = self.sess.run([self.d_cross_entropy, self.d_train_step], 
                    feed_dict={self.d_x: batch_xs, self.d_y_: batch_ys, self.d_keep: 0.5})
                print cost[0], "cost"


    if self.test :
        d_correct_prediction = tf.equal(self.d_y_out, tf.argmax(self.d_y_,1))
        #d_correct_prediction = tf.equal(tf.argmax(self.d_y , 1), tf.argmax(self.d_y_, 1))

        d_accuracy = tf.reduce_mean(tf.cast(d_correct_prediction, tf.float32))

        if self.use_loader : self.get_nn_next_test(self.batchsize)
        print(self.sess.run([d_accuracy, self.d_cross_entropy], 
            feed_dict={self.d_x: self.mnist_test.images, self.d_y_: self.mnist_test.labels, self.d_keep: 1.0}))

    if self.predict_dot :
        for i in range(start, stop ) :
            batch_0, batch_1 = self.get_nn_next_predict(self.batchsize)
            if len(batch_0) > 0 :
                out.extend( self.sess.run([self.d_y_out, self.d_cross_entropy], 
                    feed_dict={self.d_x : batch_0, self.d_y_: batch_1, self.d_keep: 1.0})[0])
                print "out" , len(out) , i, self.cursor_tot, out[:10],"..."

编辑我已经对这个问题中的代码进行了重要的编辑。非常感谢vijay m让我走这么远。任何帮助都将不胜感激。谢谢。你知道吗

Answer 1

这段代码中的问题是对输入调用dropout。你的是单层网络，你不需要dropout。使用像Adam这样的动量优化器进行更快的训练。我所做的改变：

d_y_logits_1 = tf.matmul(d_x, d_W_1) + d_b_1
d_y_mid = tf.nn.relu(d_y_logits_1) 

d_y_logits_2 = tf.matmul(d_y_mid, d_W_2) + d_b_2

d_y_softmax = tf.nn.softmax_cross_entropy_with_logits(logits=d_y_logits_2, labels=d_y_)

d_cross_entropy = tf.reduce_mean(d_y_softmax)

d_train_step = tf.train.AdamOptimizer(0.01).minimize(d_cross_entropy)