这个TensorFlow示例实际上如何更新权重以找到解决方案

Question

tensorflow、python和numpy的新手（我想这就是本示例中的全部内容）

在下面的代码中我（几乎）理解了更新_重量.跑步（）循环调用正在计算损失并开发新的权重。我不明白的是，这实际上是如何导致权重改变的。在

我被卡住的地方被评论了这是我不明白的

更新之间的关系是什么_重量.跑步（）新的值被放在权重中？-或者也许；为什么什么时候重量.评估在值已更改的循环后调用？在

谢谢你的帮助

#@test {"output": "ignore"}

# Import tf
import tensorflow as tf

# Numpy is Num-Pie n dimensional arrays
# https://en.wikipedia.org/wiki/NumPy
import numpy as np

# Plotting library
# http://matplotlib.org/users/pyplot_tutorial.html
import matplotlib.pyplot as plt

# %matplotlib magic
# http://ipython.readthedocs.io/en/stable/interactive/tutorial.html#magics-explained
%matplotlib inline

# Set up the data with a noisy linear relationship between X and Y.
# Variable?
num_examples = 5
noise_factor = 1.5
line_x_range = (-10,10)

#Just variables in Python
# np.linspace - Return evenly spaced numbers over a specified interval.
X = np.array([
        np.linspace(line_x_range[0], line_x_range[1], num_examples), 
        np.linspace(line_x_range[0], line_x_range[1], num_examples)
    ])

# Plot out the starting data
# plt.figure(figsize=(4,4))
# plt.scatter(X[0], X[1])
# plt.show()

# npm.random.randn - Return a sample (or samples) from the “standard normal” distribution.
# Generate noise for x and y (2)
noise = np.random.randn(2, num_examples) * noise_factor

# plt.figure(figsize=(4,4))
# plt.scatter(noise[0],noise[1])
# plt.show()

# += on an np.array
X += noise

# The 'Answer' polyfit to the noisy data
answer_m, answer_b = np.polyfit(X[0], X[1], 1)


# Destructuring Assignment - http://codeschool.org/python-additional-miscellany/
x, y = X

# plt.figure(figsize=(4,4))
# plt.scatter(x, y)
# plt.show()

# np.array
# for a in x
#  [(1., a) for a in [1,2,3]] => [(1.0, 1), (1.0, 2), (1.0, 3)]
# numpy.ndarray.astype - http://docs.scipy.org/doc/numpy/reference/generated/numpy.ndarray.astype.html 
# Copy of the array, cast to a specified type.
x_with_bias = np.array([(1., a) for a in x]).astype(np.float32)

#Just variables in Python
# The difference between our current outputs and the training outputs over time
# Starts high and decreases
losses = []
history = []
training_steps = 50
learning_rate = 0.002

# Start the session and give it a variable name sess 
with tf.Session() as sess:
  # Set up all the tensors, variables, and operations.
  # Creates a constant tensor
  input = tf.constant(x_with_bias)
  # Transpose the ndarray y of random float numbers
  target = tf.constant(np.transpose([y]).astype(np.float32))
  # Start with random weights
  weights = tf.Variable(tf.random_normal([2, 1], 0, 0.1))

  # Initialize variables ...?obscure?
  tf.initialize_all_variables().run()
  print('Initialization complete')

  # tf.matmul - Matrix Multiplication
  # What are yhat? Why this name?
  yhat = tf.matmul(input, weights)

  # tf.sub - Matrix Subtraction
  yerror = tf.sub(yhat, target)

  # tf.nn.l2_loss - Computes half the L2 norm of a tensor without the sqrt
  # loss function?
  loss = tf.nn.l2_loss(yerror)

  # tf.train.GradientDescentOptimizer - Not sure how this is updating the weights tensor?
  # What is it operating on?
  update_weights = tf.train.GradientDescentOptimizer(learning_rate).minimize(loss)

  # _ in Python is conventionally used for a throwaway variable
  for step in range(training_steps):
    # Repeatedly run the operations, updating the TensorFlow variable.
    # THIS IS WHAT I DONT UNDERSTAND
    update_weights.run()
    losses.append(loss.eval())
    b, m = weights.eval()
    history.append((b,m,step))

  # Training is done, get the final values for the graphs
  betas = weights.eval()
  yhat = yhat.eval()

# Show the fit and the loss over time.
# destructuring assignment
fig, (ax1, ax2, ax3) = plt.subplots(1, 3)

# Adjust whitespace between plots
plt.subplots_adjust(wspace=.2)

# Output size of the figure
fig.set_size_inches(12, 4)

ax1.set_title("Final Data Fit")
ax1.axis('equal')
ax1.axis([-15, 15, -15, 15])

# Scatter plot data x and y (pairs?) set with 60% opacity
ax1.scatter(x, y, alpha=.6)
# Scatter plot x and np.transpose(yhat)[0] (must be same length), in red, 50% transparency
# these appear to be the x values mapped onto the 
ax1.scatter(x, np.transpose(yhat)[0], c="r", alpha=.5)

# Add the line along the slope defined by betas (whatever that is)
ax1.plot(line_x_range, [betas[0] + a * betas[1] for a in line_x_range], "g", alpha=0.6)

# This polyfit coefficients are reversed in order vs the betas
ax1.plot(line_x_range, [answer_m * a + answer_b for a in line_x_range], "r", alpha=0.3)


ax2.set_title("Loss over Time")

# Create a range of intefers from 0 to training_steps and plot the losses as a curve
ax2.plot(range(0, training_steps), losses)

ax2.set_ylabel("Loss")
ax2.set_xlabel("Training steps")

ax3.set_title("Slope over Time")
ax3.axis('equal')
ax3.axis([-15, 15, -15, 15])

for b, m, step in history:
  ax3.plot(line_x_range, [b + a * m for a in line_x_range], "g", alpha=0.2)

# This line seems to be superfluous removing it doesn't change the behaviour
plt.show()

Answer 1

所以update_weights（）正在对您定义为预测和目标之间的错误的损失调用一个最小值。在

它将做的是在权重中添加一些小的量（由学习速率参数控制的小数量），使你的损失减少，从而使你的预测“更真实”。在

这是当您调用update_weights（）时发生的情况，因此在调用之后，您的权重从一个很小的值变为原来的值，并且如果一切按照计划进行，您的损失值就会减少。在

如果你真的想要改变权重的话，或者你想看看你的减肥算法是怎么做的。在

通过想象损失是如何变化的，你可以获得很多洞察力。 这就是为什么你必须看到参数和损失演变的全部历史；这就是为什么你要在每一步评估它们。在

eval或run操作不同于在minimizer上执行的时候和在参数上的操作在minimizer上执行时，它将将应用于权重，当您对权重进行简单的求值时。 我强烈建议你阅读this website，作者比我解释得更好，更详细。在