培训数据所需的时间。LSTM公司

Question

我的数据集包含1000个用户10天的数据。我正在为每个用户训练和测试数据，以提高预测精度。问题是对于第一个用户培训100个历元需要5 secs，而对于第100个用户，100个历元需要超过five minutes。每个用户的培训时间都在增加。如何减少训练时间>；由于位置点是分类的，因此对位置点进行一次热编码。你知道吗

list = list_users[:100]
with open("accuracy_Lstm.csv","w") as f:
    f.write('user,LSTM \n')
    for user in list:
        user_data = newdataframe[newdataframe.user==user]
        encoded=encoding(user_data)
        X_train = []
        y_train = []
        for i in range(1, len(encoded)-96):
            X_train.append(encoded[i-1])
            y_train.append(encoded[i])
        X_train, y_train = np.array(X_train), np.array(y_train)
        X_test = encoded[-192:-96,:]
        X_true = encoded[-96:,:]
        X_trainL=X_train.reshape(X_train.shape[0],1,X_train.shape[1])
        time_steps = 1
    #Lstm
        model = Sequential()
        model.add(LSTM(X_train.shape[1], input_shape=(time_steps,X_train.shape[1]), activation='relu'))
        model.add(Dense(X_train.shape[1]))
        model.compile(loss='mse', optimizer='adam')
        model.fit(X_trainL, y_train, batch_size=96, epochs=100, verbose =1)
        model.summary()

        X_testL=X_test.reshape(X_test.shape[0],1,X_test.shape[1])

        pedL =one_hot_decode(model.predict(X_testL))
        true=one_hot_decode(X_true)
        try:
            accuracy = ((sum(x == y for x, y in zip(pedL, true)))/(len(pedL)))*100
        except ZeroDivisionError:
            accuracy = 0
        f.write(' %d,  %f \n'%(user, accuracy))

如何减少用户的培训时间？你知道吗

Answer 1

问题是，在for循环的每次迭代中，您都在重新创建一个新模型。对于每个模型，这都是相当占用内存的，应该避免。这就是为什么第一个模型的训练速度相当快，然后每一个新模型的速度较慢。即使每次迭代del model也没有帮助，因为内存泄漏是在tensorflow中发生的。您可以在for循环的开头清除会话，但这本身就相当慢。你知道吗

我建议您在循环之外创建模型，然后在每次迭代时重新初始化模型的权重（因为模型体系结构在迭代之间不会改变）

---

编辑：

正如在评论中所建议的，下面是如何实现这个想法的示例代码。由于问题本身不是一个正常工作的代码，我没有测试和验证下面的代码是否正常工作而没有内存泄漏，但是根据我以前的经验，我认为应该这样做。你知道吗

list = list_users[:100]

def get_model():
    model = Sequential()
    model.add(LSTM(X_train.shape[1], input_shape=(time_steps,X_train.shape[1]), activation='relu'))
    model.add(Dense(X_train.shape[1]))
    model.compile(loss='mse', optimizer='adam')
    model.summary()
    return(model, model.get_weights())

with open("accuracy_Lstm.csv","w") as f:
    f.write('user,LSTM \n')
    model, initial_weights = get_model()
    for user in list:
        user_data = newdataframe[newdataframe.user==user]
        encoded=encoding(user_data)
        X_train = []
        y_train = []
        for i in range(1, len(encoded)-96):
            X_train.append(encoded[i-1])
            y_train.append(encoded[i])
        X_train, y_train = np.array(X_train), np.array(y_train)
        X_test = encoded[-192:-96,:]
        X_true = encoded[-96:,:]
        X_trainL=X_train.reshape(X_train.shape[0],1,X_train.shape[1])
        time_steps = 1
    #Lstm
        model.set_weights(initial_weights)
        model.fit(X_trainL, y_train, batch_size=96, epochs=100, verbose =1)

        X_testL=X_test.reshape(X_test.shape[0],1,X_test.shape[1])

        pedL =one_hot_decode(model.predict(X_testL))
        true=one_hot_decode(X_true)
        try:
            accuracy = ((sum(x == y for x, y in zip(pedL, true)))/(len(pedL)))*100
        except ZeroDivisionError:
            accuracy = 0
        f.write(' %d,  %f \n'%(user, accuracy))

---

编辑2:

由于每个用户的模型形状都在变化，因此需要在每次迭代时创建模型。因此，上述解决方案将不起作用。 您可以通过tf.keras.backend.clear_session()清除tensorflow内存，但是它非常慢（这就是我在上面的解决方案中尝试避免它的原因）。下面的解决方案对于每个用户可能会慢于5秒（由于增加了清除时间），但是对于每个已使用的用户应该是恒定的，这与您拥有的用户数无关。你知道吗

list = list_users[:100]

def get_model(input_size):
    # clear the memory of tensorflow before creating a new model
    tf.keras.backend.clear_session()
    # creare new model
    model = Sequential()
    model.add(LSTM(input_size, input_shape=(time_steps,input_size), activation='relu'))
    model.add(Dense(input_size))
    model.compile(loss='mse', optimizer='adam')
    model.summary()
    return(model)

with open("accuracy_Lstm.csv","w") as f:
    f.write('user,LSTM \n')
    for user in list:
        # clear the memory of tensorflow at each new model
        tf.keras.backend.clear_session()

        user_data = newdataframe[newdataframe.user==user]
        encoded=encoding(user_data)
        X_train = []
        y_train = []
        for i in range(1, len(encoded)-96):
            X_train.append(encoded[i-1])
            y_train.append(encoded[i])
        X_train, y_train = np.array(X_train), np.array(y_train)
        X_test = encoded[-192:-96,:]
        X_true = encoded[-96:,:]
        X_trainL=X_train.reshape(X_train.shape[0],1,X_train.shape[1])
        time_steps = 1
    #Lstm
        model = get_model(X_train.shape[1])
        model.fit(X_trainL, y_train, batch_size=96, epochs=100, verbose =1)

        X_testL=X_test.reshape(X_test.shape[0],1,X_test.shape[1])

        pedL =one_hot_decode(model.predict(X_testL))
        true=one_hot_decode(X_true)
        try:
            accuracy = ((sum(x == y for x, y in zip(pedL, true)))/(len(pedL)))*100
        except ZeroDivisionError:
            accuracy = 0
        f.write(' %d,  %f \n'%(user, accuracy))