保存的Keras模型在运行每个tim时给出不同的结果

2024-04-26 17:24:21 发布

您现在位置:Python中文网/ 问答频道 /正文

嗨,我正在训练一个异常自动编码器模型检测。这里是我的培训代码

def model_save(i,start,end):

    df_normal = pd.read_csv("/home/ram/Downloads/19_channel_2.csv")
    df_normal = df_normal.drop(['Unnamed: 0'],axis=1)
    len_values = len((df_normal))
    #print(len_values)
    point = ((len_values))%18
    if point==0:
        df_normal = pd.DataFrame(df_normal.iloc[:,start:end])
    else:
        df_normal = pd.DataFrame(df_normal.iloc[:-point,start:end])


    #df_normal = pd.DataFrame(df_normal.iloc[:-point,:])
    df_normal = df_normal.T
    df_normal = (df_normal.values).reshape(-1,18)
    df_normal = pd.DataFrame(df_normal)


    RANDOM_SEED = 101
    X_train = df_normal.values
    print('Training data size   :', X_train.shape)
    scaler = MinMaxScaler()
    X_train_scaled = scaler.fit_transform(X_train)
    input_dim = X_train.shape[1]
    encoding_dim = 6



    input_layer = Input(shape=(input_dim, ))
    encoder = Dense(encoding_dim, activation="tanh",activity_regularizer=regularizers.l1(10e-5))(input_layer)
    encoder = Dense(int(encoding_dim / 2), activation="tanh")(encoder)
    encoder = Dense(int(2), activation="tanh")(encoder)
    decoder = Dense(int(encoding_dim/ 2), activation='tanh')(encoder)
    decoder = Dense(int(encoding_dim), activation='tanh')(decoder)
    decoder = Dense(input_dim, activation='tanh')(decoder)
    autoencoder = Model(inputs=input_layer, outputs=decoder)
    #autoencoder.summary()
    nb_epoch = 100
    batch_size = 50
    autoencoder.compile(optimizer='adam', loss='mse' )

    t_ini = datetime.datetime.now()
    autoencoder.fit(X_train_scaled, X_train_scaled,
                            epochs=nb_epoch,
                            batch_size=batch_size,
                            shuffle=True,
                            validation_split=0.1,
                            verbose=0
                            )

    t_fin = datetime.datetime.now()
    print('Time to run the model: {} Sec.'.format((t_fin - t_ini).total_seconds()))
    autoencoder.save("/home/ram/Downloads/AutoEncoderModels/Keras/auto_encoder_model_19H_C2_zone_pos_" + "S" + str(start) + '_' + 'E' + str(end) + '.h5' )
    print("Saved model " + str(i) + " to disk")
    with open("/home/ram/Downloads/AutoEncoderScalars/auto_encoder_scalar_19H_C2_zone_pos_" + "S" + str(start) + '_' + 'E' + str(end) + ".pkl" , "wb") as outfile:
        pkl.dump(scaler, outfile)
    print("Saved scalar " + str(i) + " to disk")
    K.clear_session()

这是我测试模型的代码

from keras.models import load_model
    import pickle as pkl
    import numpy as np
    import json
    import pandas as pd
def reconstruction_error(i,X_test_scaled,autoencoder):
    predictions = autoencoder.predict(X_test_scaled)
    print(predictions)
    mse = np.mean(np.power(X_test_scaled - predictions, 2), axis=1)
    df_error = pd.DataFrame({'reconstruction_error': mse})
    #print(df_error)
    outliers = df_error.index[df_error.reconstruction_error > 0.1].tolist()
    #print(outliers)
    print("The no of anomalies for zone" + str(i) + "is" + str(len(outliers)))
    global sum_outliers
    sum_outliers += len(outliers)
    print(sum_outliers)
def model_run(i,start,end):
    df_test_2 = pd.DataFrame(df_temp.iloc[:,start:end])
    df_test_T = df_test_2.T
    df_test_reshaped = df_test_T.values.reshape(-1,5)
    autoencoder = load_model('/home/ram/Downloads/AutoEncoderModels/Keras/auto_encoder_model_19H_C2_zone_pos_S' + str(start) + '_' + 'E' + str(end) + '.h5')
    with open("/home/ram/Downloads/AutoEncoderScalars/zone" + str(i) + ".pkl", "rb") as infile:
        scaler = pkl.load(infile)
        X_test_scaled =scaler.transform(df_test_reshaped)
        reconstruction_error(i,X_test_scaled,autoencoder)

当我在不同的计算机上运行上述测试数据代码时,每次接收到的重建误差是不同的,我每次训练数据时都设置了随机种子常数模特。但是它如何在测试数据上给出不同的结果?到目前为止,我看到的所有答案都要求我在培训中更改代码代码。但是在这里,当我在测试数据上运行它时,我得到了不同的结果

我在网上尝试了各种答案,比如shuffle=False,手动变量初始化(True)

但似乎都不管用


Tags: testdfencodermodeltrainerrorstartend