概述
如果任何神经元的权重矩阵的L2范数超过max_norm,我想缩减其权重,使其L2范数正好是max_norm.
使用TensorFlow表达这一点的最佳方式是什么？
import tensorflow as tf
def maxnorm_regularizer(threshold,axes=1,name="maxnorm",collection="maxnorm"):
def maxnorm(weights):
clipped = tf.clip_by_norm(weights,clip_norm=threshold,axes=axes)
clip_weights = tf.assign(weights,clipped,name=name)
tf.add_to_collection(collection,clip_weights)
return None # there is no regularization loss term
return maxnorm以下是您将如何使用它：
from tensorflow.contrib.layers import fully_connected
from tensorflow.contrib.framework import arg_scope
with arg_scope(
[fully_connected],weights_regularizer=max_norm_regularizer(1.5)):
hidden1 = fully_connected(X,200,scope="hidden1")
hidden2 = fully_connected(hidden1,100,scope="hidden2")
outputs = fully_connected(hidden2,5,activation_fn=None,scope="outs")
max_norm_ops = tf.get_collection("max_norm")
[...]
with tf.Session() as sess:
sess.run(init)
for epoch in range(n_epochs):
for X_batch,y_batch in load_next_batch():
sess.run(training_op,Feed_dict={X: X_batch,y: y_batch})
sess.run(max_norm_ops)这将创建一个3层神经网络,并在每一层(阈值为1.5)对其进行最大范数正则化训练.我只是尝试过,似乎工作.希望这可以帮助！欢迎提出改进建议.
总结
以上是编程之家为你收集整理的python – 使用TensorFlow重整化权重矩阵全部内容，希望文章能够帮你解决python – 使用TensorFlow重整化权重矩阵所遇到的程序开发问题。
如果您也喜欢它,动动您的小指点个赞吧
602392714
清零编程群