张量的前n个值为1，其他值为0

Question

我正在尝试为keras模型编写一个自定义度量，以用于推荐模型。我需要找到一种方法将预测向量转换为预测中的前2个元素为1，其他元素为0的向量，然后将其与真值进行比较，并计算它们的（#交点）/（#并集） 所以我们假设模型的预测是：

pred=[0.01,0.3,0,40.01,0.01,0.2,0.02,0.05]

前两个值分别为0.3和0.4，索引为1和2。那么我建议：

reccomend=[0,1,1,0,0,0,0,0]

如果真值如下所示，那么它们的交集只是索引2，它们的并集是[1,2,3]，那么我应该返回1/3

真理=[0,0,1,1,0,0,0,0]

我用numpy数组做了这个，它在模型训练后工作，但我找不到一种方法用张量做这个，所以我不能在模型中使用它作为度量

Numpy代码：

y_pred = np.zeros(len(predictions))
ind = predictions.argsort()[-2:][::-1]
y_pred[ind] = 1
def dice_index(y_true, y_pred):
    innerproduct = np.inner(y_true, y_pred)
    union = np.sum(y_true)+np.sum(y_pred) - innerproduct
    return innerproduct/union

我按照建议试过：

def dice_index_metric(y_true, y_pred):
    ind = tf.argsort(y_pred,axis=-1,direction='ASCENDING',stable=False,name=None)[-2:]
    ind = ind[..., tf.newaxis]
    updates = tf.constant([1, 1])
    y_pred1 = tf.scatter_nd(ind, updates, tf.shape(y_pred))
    innerproduct = tf.minimum(y_true, y_pred1)
    innerproduct = tf.reduce_sum(innerproduct)
    union= tf.maximum(y_true, y_pred1)
    union = tf.reduce_sum(union)
    return innerproduct/union

然后跑

model.compile(optimizer= keras.optimizers.Adam(learning_rate = 1e-3), loss = inner_product, metrics=dice_index_metric)
 model.fit([X_train], [y_train], epochs=50, batch_size = 1, validation_split=0.2, 
                 callbacks = keras.callbacks.EarlyStopping(monitor="val_loss", min_delta=0, patience=5), verbose=2)

其错误如下：

    ---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-38-270dbe25d468> in <module>
----> 1 hist = model.fit([X_train], [y_train], epochs=50, batch_size = 1, validation_split=0.2, 
      2                  callbacks = keras.callbacks.EarlyStopping(monitor="val_loss", min_delta=0, patience=5), verbose=2)

~\AppData\Roaming\Python\Python38\site-packages\tensorflow\python\keras\engine\training.py in _method_wrapper(self, *args, **kwargs)
    106   def _method_wrapper(self, *args, **kwargs):
    107     if not self._in_multi_worker_mode():  # pylint: disable=protected-access
--> 108       return method(self, *args, **kwargs)
    109 
    110     # Running inside `run_distribute_coordinator` already.

~\AppData\Roaming\Python\Python38\site-packages\tensorflow\python\keras\engine\training.py in fit(self, x, y, batch_size, epochs, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch, steps_per_epoch, validation_steps, validation_batch_size, validation_freq, max_queue_size, workers, use_multiprocessing)
   1096                 batch_size=batch_size):
   1097               callbacks.on_train_batch_begin(step)
-> 1098               tmp_logs = train_function(iterator)
   1099               if data_handler.should_sync:
   1100                 context.async_wait()

~\AppData\Roaming\Python\Python38\site-packages\tensorflow\python\eager\def_function.py in __call__(self, *args, **kwds)
    778       else:
    779         compiler = "nonXla"
--> 780         result = self._call(*args, **kwds)
    781 
    782       new_tracing_count = self._get_tracing_count()

~\AppData\Roaming\Python\Python38\site-packages\tensorflow\python\eager\def_function.py in _call(self, *args, **kwds)
    821       # This is the first call of __call__, so we have to initialize.
    822       initializers = []
--> 823       self._initialize(args, kwds, add_initializers_to=initializers)
    824     finally:
    825       # At this point we know that the initialization is complete (or less

~\AppData\Roaming\Python\Python38\site-packages\tensorflow\python\eager\def_function.py in _initialize(self, args, kwds, add_initializers_to)
    694     self._graph_deleter = FunctionDeleter(self._lifted_initializer_graph)
    695     self._concrete_stateful_fn = (
--> 696         self._stateful_fn._get_concrete_function_internal_garbage_collected(  # pylint: disable=protected-access
    697             *args, **kwds))
    698 

~\AppData\Roaming\Python\Python38\site-packages\tensorflow\python\eager\function.py in _get_concrete_function_internal_garbage_collected(self, *args, **kwargs)
   2853       args, kwargs = None, None
   2854     with self._lock:
-> 2855       graph_function, _, _ = self._maybe_define_function(args, kwargs)
   2856     return graph_function
   2857 

~\AppData\Roaming\Python\Python38\site-packages\tensorflow\python\eager\function.py in _maybe_define_function(self, args, kwargs)
   3211 
   3212       self._function_cache.missed.add(call_context_key)
-> 3213       graph_function = self._create_graph_function(args, kwargs)
   3214       self._function_cache.primary[cache_key] = graph_function
   3215       return graph_function, args, kwargs

~\AppData\Roaming\Python\Python38\site-packages\tensorflow\python\eager\function.py in _create_graph_function(self, args, kwargs, override_flat_arg_shapes)
   3063     arg_names = base_arg_names + missing_arg_names
   3064     graph_function = ConcreteFunction(
-> 3065         func_graph_module.func_graph_from_py_func(
   3066             self._name,
   3067             self._python_function,

~\AppData\Roaming\Python\Python38\site-packages\tensorflow\python\framework\func_graph.py in func_graph_from_py_func(name, python_func, args, kwargs, signature, func_graph, autograph, autograph_options, add_control_dependencies, arg_names, op_return_value, collections, capture_by_value, override_flat_arg_shapes)
    984         _, original_func = tf_decorator.unwrap(python_func)
    985 
--> 986       func_outputs = python_func(*func_args, **func_kwargs)
    987 
    988       # invariant: `func_outputs` contains only Tensors, CompositeTensors,

~\AppData\Roaming\Python\Python38\site-packages\tensorflow\python\eager\def_function.py in wrapped_fn(*args, **kwds)
    598         # __wrapped__ allows AutoGraph to swap in a converted function. We give
    599         # the function a weak reference to itself to avoid a reference cycle.
--> 600         return weak_wrapped_fn().__wrapped__(*args, **kwds)
    601     weak_wrapped_fn = weakref.ref(wrapped_fn)
    602 

~\AppData\Roaming\Python\Python38\site-packages\tensorflow\python\framework\func_graph.py in wrapper(*args, **kwargs)
    971           except Exception as e:  # pylint:disable=broad-except
    972             if hasattr(e, "ag_error_metadata"):
--> 973               raise e.ag_error_metadata.to_exception(e)
    974             else:
    975               raise

ValueError: in user code:

    C:\Users\haluk\AppData\Roaming\Python\Python38\site-packages\tensorflow\python\keras\engine\training.py:806 train_function  *
        return step_function(self, iterator)
    <ipython-input-36-2c54f0983574>:5 dice_index_metric  *
        y_pred1 = tf.scatter_nd(ind, updates, tf.shape(y_pred))
    C:\Users\haluk\AppData\Roaming\Python\Python38\site-packages\tensorflow\python\ops\gen_array_ops.py:8855 scatter_nd  **
        _, _, _op, _outputs = _op_def_library._apply_op_helper(
    C:\Users\haluk\AppData\Roaming\Python\Python38\site-packages\tensorflow\python\framework\op_def_library.py:742 _apply_op_helper
        op = g._create_op_internal(op_type_name, inputs, dtypes=None,
    C:\Users\haluk\AppData\Roaming\Python\Python38\site-packages\tensorflow\python\framework\func_graph.py:591 _create_op_internal
        return super(FuncGraph, self)._create_op_internal(  # pylint: disable=protected-access
    C:\Users\haluk\AppData\Roaming\Python\Python38\site-packages\tensorflow\python\framework\ops.py:3477 _create_op_internal
        ret = Operation(
    C:\Users\haluk\AppData\Roaming\Python\Python38\site-packages\tensorflow\python\framework\ops.py:1974 __init__
        self._c_op = _create_c_op(self._graph, node_def, inputs,
    C:\Users\haluk\AppData\Roaming\Python\Python38\site-packages\tensorflow\python\framework\ops.py:1815 _create_c_op
        raise ValueError(str(e))

    ValueError: The outer 2 dimensions of indices.shape=[1,11,1] must match the outer 2 dimensions of updates.shape=[2]: Shapes must be equal rank, but are 2 and 1 for '{{node ScatterNd}} = ScatterNd[T=DT_INT32, Tindices=DT_INT32](strided_slice_2, Const_3, Shape_1)' with input shapes: [1,11,1], [2], [2].

Answer 1

我发现解决方案如下：

def dice_index_metric(y_true, y_pred):
    second_max = tf.sort(y_pred,axis=-1,direction='ASCENDING')[:,-2,tf.newaxis]
    y_pred1 = tf.cast(y_pred>=second_max, tf.float32)
    innerproduct = tf.minimum(y_true, y_pred1)
    innerproduct = tf.reduce_sum(innerproduct)
    union= tf.maximum(y_true, y_pred1)
    union = tf.reduce_sum(union)
    return innerproduct/union

Answer 2

请尝试以下代码：

import tensorflow as tf
pred = tf.constant([0.01, 0.3, 0.4, 0.01, 0.01, 0.2, 0.02, 0.05])
y_true = [0, 0, 1, 1, 0, 0, 0, 0]
ind = tf.argsort(pred,axis=-1,direction='ASCENDING',stable=False,name=None)[-2:]
ind = ind[..., tf.newaxis]
updates = tf.constant([1, 1])
y_pred = tf.scatter_nd(ind, updates, tf.shape(pred))
def dice_index(y_true, y_pred):
  innerproduct = tf.minimum(y_true, y_pred)
  innerproduct = tf.reduce_sum(innerproduct)
  union = tf.maximum(y_true, y_pred)
  union = tf.reduce_sum(union)
  return innerproduct/union
a = dice_index(y_true, y_pred)