这是一个厚脸皮的单线滥用broadcasted
比较-
(np.arange(a.max()) == a[...,None]-1).astype(int)
样品运行-
In [120]: a
Out[120]:
array([[1, 7, 5, 3],
[2, 4, 1, 4]])
In [121]: (np.arange(a.max()) == a[...,None]-1).astype(int)
Out[121]:
array([[[1, 0, 0, 0, 0, 0, 0],
[0, 0, 0, 0, 0, 0, 1],
[0, 0, 0, 0, 1, 0, 0],
[0, 0, 1, 0, 0, 0, 0]],
[[0, 1, 0, 0, 0, 0, 0],
[0, 0, 0, 1, 0, 0, 0],
[1, 0, 0, 0, 0, 0, 0],
[0, 0, 0, 1, 0, 0, 0]]])
对于0-based
索引,将是-
In [122]: (np.arange(a.max()+1) == a[...,None]).astype(int)
Out[122]:
array([[[0, 1, 0, 0, 0, 0, 0, 0],
[0, 0, 0, 0, 0, 0, 0, 1],
[0, 0, 0, 0, 0, 1, 0, 0],
[0, 0, 0, 1, 0, 0, 0, 0]],
[[0, 0, 1, 0, 0, 0, 0, 0],
[0, 0, 0, 0, 1, 0, 0, 0],
[0, 1, 0, 0, 0, 0, 0, 0],
[0, 0, 0, 0, 1, 0, 0, 0]]])
如果单项编码要覆盖从最小值到最大值的值范围,请偏移最小值,然后将其馈送到建议的0-based
索引方法中。这也将适用于本文后面稍后讨论的其余方法。
这是在同一样本上运行的示例-
In [223]: a
Out[223]:
array([[ 6, 12, 10, 8],
[ 7, 9, 6, 9]])
In [224]: a_off = a - a.min() # Feed a_off to proposed approaches
In [225]: (np.arange(a_off.max()+1) == a_off[...,None]).astype(int)
Out[225]:
array([[[1, 0, 0, 0, 0, 0, 0],
[0, 0, 0, 0, 0, 0, 1],
[0, 0, 0, 0, 1, 0, 0],
[0, 0, 1, 0, 0, 0, 0]],
[[0, 1, 0, 0, 0, 0, 0],
[0, 0, 0, 1, 0, 0, 0],
[1, 0, 0, 0, 0, 0, 0],
[0, 0, 0, 1, 0, 0, 0]]])
如果您可以使用带有True
for1's
和False for的布尔数组0's
,则可以跳过.astype(int)
转换。
我们还可以初始化零数组,并使用索引到输出中advanced- indexing
。因此,为了0-based
建立索引,我们将-
def onehot_initialization(a):
ncols = a.max()+1
out = np.zeros(a.shape + (ncols,), dtype=int)
out[all_idx(a, axis=2)] = 1
return out
辅助功能-
# https://stackoverflow.com/a/46103129/ @Divakar
def all_idx(idx, axis):
grid = np.ogrid[tuple(map(slice, idx.shape))]
grid.insert(axis, idx)
return tuple(grid)
当处理更大范围的值时,这应该特别有效。
对于1-based
索引,只需a-1
输入作为输入即可。
现在,如果您正在寻找稀疏数组作为输出和AFAIK,因为scipy的内置稀疏矩阵仅支持2D
格式,则可以得到稀疏输出,该输出是前面显示的输出的重塑版本,其中前两个轴合并而第三个轴保持不变完整。0-based
索引的实现看起来像这样-
from scipy.sparse import coo_matrix
def onehot_sparse(a):
N = a.size
L = a.max()+1
data = np.ones(N,dtype=int)
return coo_matrix((data,(np.arange(N),a.ravel())), shape=(N,L))
同样,为1-based
建立索引,只需将其a-1
作为输入即可。
样品运行-
In [157]: a
Out[157]:
array([[1, 7, 5, 3],
[2, 4, 1, 4]])
In [158]: onehot_sparse(a).toarray()
Out[158]:
array([[0, 1, 0, 0, 0, 0, 0, 0],
[0, 0, 0, 0, 0, 0, 0, 1],
[0, 0, 0, 0, 0, 1, 0, 0],
[0, 0, 0, 1, 0, 0, 0, 0],
[0, 0, 1, 0, 0, 0, 0, 0],
[0, 0, 0, 0, 1, 0, 0, 0],
[0, 1, 0, 0, 0, 0, 0, 0],
[0, 0, 0, 0, 1, 0, 0, 0]])
In [159]: onehot_sparse(a-1).toarray()
Out[159]:
array([[1, 0, 0, 0, 0, 0, 0],
[0, 0, 0, 0, 0, 0, 1],
[0, 0, 0, 0, 1, 0, 0],
[0, 0, 1, 0, 0, 0, 0],
[0, 1, 0, 0, 0, 0, 0],
[0, 0, 0, 1, 0, 0, 0],
[1, 0, 0, 0, 0, 0, 0],
[0, 0, 0, 1, 0, 0, 0]])
情况1 :
In [160]: a = np.random.randint(0,100,(100,100))
In [161]: %timeit (np.arange(a.max()+1) == a[...,None]).astype(int)
1000 loops, best of 3: 1.51 ms per loop
In [162]: %timeit onehot_initialization(a)
1000 loops, best of 3: 478 µs per loop
In [163]: %timeit onehot_sparse(a)
10000 loops, best of 3: 87.5 µs per loop
In [164]: %timeit onehot_sparse(a).toarray()
1000 loops, best of 3: 530 µs per loop
案例2:
In [166]: a = np.random.randint(0,500,(100,100))
In [167]: %timeit (np.arange(a.max()+1) == a[...,None]).astype(int)
100 loops, best of 3: 8.51 ms per loop
In [168]: %timeit onehot_initialization(a)
100 loops, best of 3: 2.52 ms per loop
In [169]: %timeit onehot_sparse(a)
10000 loops, best of 3: 87.1 µs per loop
In [170]: %timeit onehot_sparse(a).toarray()
100 loops, best of 3: 2.67 ms per loop
为了获得最佳性能,我们可以修改方法2,以在2D
成形的输出数组上使用索引,还可以使用uint8
dtype来提高内存效率,从而加快分配速度,例如:
def onehot_initialization_v2(a):
ncols = a.max()+1
out = np.zeros( (a.size,ncols), dtype=np.uint8)
out[np.arange(a.size),a.ravel()] = 1
out.shape = a.shape + (ncols,)
return out
时间-
In [178]: a = np.random.randint(0,100,(100,100))
In [179]: %timeit onehot_initialization(a)
...: %timeit onehot_initialization_v2(a)
...:
1000 loops, best of 3: 474 µs per loop
10000 loops, best of 3: 128 µs per loop
In [180]: a = np.random.randint(0,500,(100,100))
In [181]: %timeit onehot_initialization(a)
...: %timeit onehot_initialization_v2(a)
...:
100 loops, best of 3: 2.38 ms per loop
1000 loops, best of 3: 213 µs per loop