什么时候调用Keras激活函数？

Question

假设：

据我所知，Keras密集层实现了以下操作（Keras docs）：

output = activation(dot(input, kernel) + bias)

我假设每次调用predict（）方法时，它都会触发激活函数运行。情况似乎并非如此

实验：

为了测试这一点，我编写了Keras自定义激活函数的两个不同实现，当激活函数运行时，应该打印到屏幕上。我特别更新了call（）函数，因为：

The call method defines the computations performed on the input. The function accepts the input tensor as its argument and returns the output tensor after applying the required operations. reference

代码：

import os
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Layer, Dense, Dropout, Activation
from tensorflow.keras.optimizers import Adam
from tensorflow.keras.activations import sigmoid
from tensorflow.keras.utils import get_custom_objects
import numpy as np

# ====================================================

# Custom Activation layer
class CustomAct(Layer):

    def __init__(self, **kwargs):
        super(CustomAct, self).__init__()

    def call(self, x):
        print ("  >> CustomAct > call()")
        return sigmoid(x)

# ====================================================

def custom_activation(x):
    print ("  >> custom_activation()")
    return sigmoid(x)

# ====================================================

print ("\nInit Datasets:")
X = np.array([[0,0],[0,1],[1,0],[1,1]])
y = np.array([[0],[1],[1],[0]])

# ====================================================

print ("\nBuild:")
model = Sequential()
model.add(Dense(8, input_dim=2))
model.add(Activation(custom_activation))
model.add(Dense(1))
model.add(CustomAct())

# ====================================================

print ("\nCompile:")
opt = Adam(lr=0.1)
model.compile(loss='binary_crossentropy', optimizer=opt)

# ====================================================

print ("\nFit:")
model.fit(X, y, batch_size=1, epochs=10, verbose=0)

# ====================================================

print ("\nEvaluate:")
model.evaluate(X, y, verbose=0)

# ====================================================

print ("\nPredict X[0]:")
output = model.predict(X[0:1])
print (output)

print ("\nPredict X[1]:")
output = model.predict(X[1:2])
print (output)

print ("\nPredict X[2]:")
output = model.predict(X[2:3])
print (output)

print ("\nPredict X[3]:")
output = model.predict(X[3:4])
print (output)

结果：

其输出如下：

Init Datasets:

Build:
  >> custom_activation()
  >> CustomAct > call()

Compile:

Fit:
  >> custom_activation()
  >> CustomAct > call()
  >> custom_activation()
  >> CustomAct > call()

Evaluate:
  >> custom_activation()
  >> CustomAct > call()

Predict X[0]:
  >> custom_activation()
  >> CustomAct > call()
[[0.4644116]]

Predict X[1]:
[[0.48512048]]

Predict X[2]:
[[0.46812934]]

Predict X[3]:
[[0.4797536]]

期望值：

这不是我所期望的。我希望每次运行预测时都能看到打印的>> custom_activation()代码。例如：

Predict X[0]:
  >> custom_activation()  # Actual output
  >> CustomAct > call()   # Actual output
[[0.4644116]]

Predict X[1]:
  >> custom_activation()  # Expected to see this
  >> CustomAct > call()   # Expected to see this
[[0.48512048]]

Predict X[2]:
  >> custom_activation()  # Expected to see this
  >> CustomAct > call()   # Expected to see this
[[0.46812934]]

Predict X[3]:
  >> custom_activation()  # Expected to see this
  >> CustomAct > call()   # Expected to see this
[[0.4797536]]

解决方法：

如果我从model.predict(X[0:1])更改为model().numpy(X[0:1])，那么每次都会调用激活函数：

代码：

print ("\nPredict X[0]:")
output = model(X[0:1]).numpy()     # Updated line
print (output)

print ("\nPredict X[1]:")
output = model(X[1:2]).numpy()     # Updated line
print (output)

print ("\nPredict X[2]:")
output = model(X[2:3]).numpy()     # Updated line
print (output)

print ("\nPredict X[3]:")
output = model(X[3:4]).numpy()     # Updated line
print (output)

输出：

Predict X[0]:
  >> custom_activation()
  >> CustomAct > call()
[[0.02330273]]

Predict X[1]:
  >> custom_activation()
  >> CustomAct > call()
[[0.91676223]]

Predict X[2]:
  >> custom_activation()
  >> CustomAct > call()
[[0.9246548]]

Predict X[3]:
  >> custom_activation()
  >> CustomAct > call()
[[0.12687866]]

问题：

为什么每次我用一个新的输入值调用model.predict()时都不调用激活函数