我为mnist数据集开发了一个3层深度的自动编码器模型,因为我只是在这个玩具数据集上练习,因为我是这个微调范例的初学者
下面是代码
from keras import layers
from keras.layers import Input, Dense
from keras.models import Model,Sequential
from keras.datasets import mnist
import numpy as np
# Deep Autoencoder
# this is the size of our encoded representations
encoding_dim = 32 # 32 floats -> compression factor 24.5, assuming the input is 784 floats
# this is our input placeholder; 784 = 28 x 28
input_img = Input(shape=(784, ))
my_epochs = 100
# "encoded" is the encoded representation of the inputs
encoded = Dense(encoding_dim * 4, activation='relu')(input_img)
encoded = Dense(encoding_dim * 2, activation='relu')(encoded)
encoded = Dense(encoding_dim, activation='relu')(encoded)
# "decoded" is the lossy reconstruction of the input
decoded = Dense(encoding_dim * 2, activation='relu')(encoded)
decoded = Dense(encoding_dim * 4, activation='relu')(decoded)
decoded = Dense(784, activation='sigmoid')(decoded)
# this model maps an input to its reconstruction
autoencoder = Model(input_img, decoded)
# Separate Encoder model
# this model maps an input to its encoded representation
encoder = Model(input_img, encoded)
# Separate Decoder model
# create a placeholder for an encoded (32-dimensional) input
encoded_input = Input(shape=(encoding_dim, ))
# retrieve the layers of the autoencoder model
decoder_layer1 = autoencoder.layers[-3]
decoder_layer2 = autoencoder.layers[-2]
decoder_layer3 = autoencoder.layers[-1]
# create the decoder model
decoder = Model(encoded_input, decoder_layer3(decoder_layer2(decoder_layer1(encoded_input))))
# Train to reconstruct MNIST digits
# configure model to use a per-pixel binary crossentropy loss, and the Adadelta optimizer
autoencoder.compile(optimizer='adadelta', loss='binary_crossentropy')
# prepare input data
(x_train, y_train), (x_test, y_test) = mnist.load_data()
# normalize all values between 0 and 1 and flatten the 28x28 images into vectors of size 784
x_train = x_train.astype('float32') / 255.
x_test = x_test.astype('float32') / 255.
x_train = x_train.reshape((len(x_train), np.prod(x_train.shape[1:])))
x_test = x_test.reshape((len(x_test), np.prod(x_test.shape[1:])))
# Train autoencoder for 50 epochs
autoencoder.fit(x_train, x_train, epochs=my_epochs, batch_size=256, shuffle=True, validation_data=(x_test, x_test),
verbose=2)
# after 100 epochs the autoencoder seems to reach a stable train/test lost value
# Visualize the reconstructed encoded representations
# encode and decode some digits
# note that we take them from the *test* set
encodedTrainImages=encoder.predict(x_train)
encoded_imgs = encoder.predict(x_test)
decoded_imgs = decoder.predict(encoded_imgs)
# From here I want to fine tune just the encoder model
model=Sequential()
model=Sequential()
for layer in encoder.layers:
model.add(layer)
model.add(layers.Flatten())
model.add(layers.Dense(20, activation='relu'))
model.add(layers.Dropout(0.5))
model.add(layers.Dense(10, activation='softmax'))
以下是我想要微调的编码器型号。
^{pr2}$问题:1
在建立了自动编码器模型后,我只想使用编码器模型并对其进行微调,以用于mnist数据集中的分类任务,但我得到了错误。在
错误:
Traceback (most recent call last):
File "C:\Users\samer\Anaconda3\envs\tensorflow-gpu\lib\site-packages\IPython\core\interactiveshell.py", line 3267, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "<ipython-input-15-528c079e5325>", line 3, in <module>
model.add(layers.Flatten())
File "C:\Users\samer\Anaconda3\envs\tensorflow-gpu\lib\site-packages\keras\engine\sequential.py", line 181, in add
output_tensor = layer(self.outputs[0])
File "C:\Users\samer\Anaconda3\envs\tensorflow-gpu\lib\site-packages\keras\engine\base_layer.py", line 414, in __call__
self.assert_input_compatibility(inputs)
File "C:\Users\samer\Anaconda3\envs\tensorflow-gpu\lib\site-packages\keras\engine\base_layer.py", line 327, in assert_input_compatibility
str(K.ndim(x)))
ValueError: Input 0 is incompatible with layer flatten_4: expected min_ndim=3, found ndim=2
问题2:
类似地,我以后会使用预先训练的模型,其中每个自动编码器都将以贪婪的方式进行训练,然后最终的模型将被微调。有人能指导我如何进一步完成这两项任务吗。在
问候
问题1
问题是,您试图将已经平坦的层变平:编码器由一维Desnse层组成,这些层具有形状
(batch_size, dim)
。在扁平层至少需要一个二维输入,即具有三维形状
(batch_size, dim1, dim2)
(例如,Conv2D层的输出),通过移除它,模型将正确构建:哪些输出:
^{pr2}$\u\
编辑:在评论中整合问题的答案
Q:我如何确保新模型将使用与先前训练过的编码器相同的权重?
答:在您的代码中,您所做的是迭代编码器内部包含的层,然后将每个层传递给
model.add()
。您在这里所做的是将引用直接传递给每个层,因此您将在新模型中拥有相同的层。以下是使用图层名称的概念证明:哪些输出:
如您所见,编码器和自动编码器中的层引用是相同的。另外,通过改变新模型内部的层名称,我们也改变了编码器对应层的层名称。有关通过引用传递的python参数的详细信息,请查看answer。在
Q:我的数据需要一个热编码吗?如果是,那怎么办?
答:你确实需要一个热编码,因为你正在处理一个多标签的分类问题。只需使用一个方便的keras函数即可完成编码:
这是指向documentation的链接。在
\u\
问题2
关于你的第二个问题,你的目标不是很清楚,但是在我看来,你想要构建一个包含几个并行自动编码器的体系结构,这些编码器专门用于不同的任务,然后通过添加一些最终的公共层来连接它们的输出。在
无论如何,到目前为止,我能做的是建议您研究一下这个guide,它解释了如何构建多输入和多输出模型,并将其用作开始定制实现的基线。在
\u\
编辑2:问题2答案整合
对于贪婪的训练任务,方法是一次训练一层,在附加新层时冻结之前的所有层。下面是一个3(+1)贪心训练层网络的示例,该网络后来用作新模型的基础:
大概就是这样,但是我必须说贪婪层训练不再是一个合适的解决方案:现在的ReLU、Dropout和其他正则化技术使得贪婪层训练成为过时和耗时的权重初始化,因此,在进行贪婪的训练之前,你可能还想看看其他的可能性。在
\u\
相关问题 更多 >
编程相关推荐