毕设笔记
1. 余弦相似度
https://zh.wikipedia.org/wiki/%E4%BD%99%E5%BC%A6%E7%9B%B8%E4%BC%BC%E6%80%A7
2. L1 and L2 norm
http://www.chioka.in/differences-between-l1-and-l2-as-loss-function-and-regularization/
3. pathlib
https://docs.python.org/3/library/pathlib.html
4. partial()
https://zhuanlan.zhihu.com/p/47124891
5. multiprocessing
https://docs.python.org/3/library/multiprocessing.html
6. tqdm
显示进度条
https://tqdm.github.io/docs/tqdm/
7. join
https://www.w3schools.com/python/ref_string_join.asp
8. typing
https://docs.python.org/3/library/typing.html
9. librosa
https://librosa.org/doc/latest/index.html
10. normalize_volume
https://dsp.stackexchange.com/questions/8785/how-to-compute-dbfs
11. mel spectrogram
https://zhuanlan.zhihu.com/p/421460202
https://wenku.baidu.com/view/8a41d8a280d049649b6648d7c1c708a1284a0a26.html
12. torch.rand torch.randn
rand（）取值从0，1之间的均匀分布中抽样。
randn（）取值从以0为均值，1为方差的标准正态分布中抽样。
13. PyTorch 中的尾随_表示该操作是原地执行的。
14. pytorch linear层
input v(m,)
out v(n,)
weight_matrix (m,n)
out=input*weight_matrix
15. biLSTM 双向LSTM
https://www.jianshu.com/p/471bdbd0170d
https://zhuanlan.zhihu.com/p/40119926
16. pytorch nn.embedding
https://www.jianshu.com/p/63e7acc5e890
17. highwaynetwork
用来增加网络的深度。增强深层网络的效果
https://en.wikipedia.org/wiki/Highway_network
https://zhuanlan.zhihu.com/p/35019701
18. rnn’s input_size and hidden_size
input:
(bs, len, input_size) assuming that batch_sirst==True
output:
(bs, len, hidden_size) assuming that batch_sirst==True
input_size is just the dim of a input data, for example, if a char is embedded into a vector which’d dim==512, then input is (bs, len, embed_dim), there, embed_dim is input_size
hidden_size is just hidden cell’s dim.
19. LSA Location Sensitive Attention
https://paperswithcode.com/method/location-sensitive-attention
https://www.zhihu.com/question/68482809/answer/264632289
https://cloud.tencent.com/developer/article/1614072
20. zoneout
zoneout是rnn 时间维度上的“dropout”，要么维持前一个时刻的hidden vector，要么按照一般的样子更新。不是指单独的cell，而是指训练时的一种trick。Dropout就是通用的一种深度学习技巧，训练时随机失活一些神经元，可以增强模型泛化抑制过拟合作用。zoneout是指随机失活一个rnncell，跳过一步。
https://www.zhihu.com/question/332535296/answer/733037609
21. WaveRNN
https://www.jianshu.com/p/b3019f2773ed
https://zhuanlan.zhihu.com/p/105788551
22. ResBlock
https://zhuanlan.zhihu.com/p/161639679
23. kernel_size=1 in conv
nn.Conv1d with a kernel size of 1 and nn.Linear give essentially the same results. The only differences are the initialization procedure and how the operations are applied (which has some effect on the speed). Note that using a linear layer should be faster as it is implemented as a simple matrix multiplication (+ adding a broadcasted bias vector)
https://stackoverflow.com/questions/55576314/conv1d-with-kernel-size-1-vs-linear-layer
24. torch.repeat
https://pytorch.org/docs/stable/generated/torch.Tensor.repeat.html
x = torch.tensor([1, 2, 3])
x.repeat(4, 2)
tensor([[ 1, 2, 3, 1, 2, 3],
[ 1, 2, 3, 1, 2, 3],
[ 1, 2, 3, 1, 2, 3],
[ 1, 2, 3, 1, 2, 3]])
x.repeat(4, 2, 1).size()
torch.Size([4, 2, 3])
以x.repeat(4, 2, 1).size()为例，由于需要repeat 4，2，1三次，但是x为一维的，所以unsqueeze两次得到x.shape=(1,1,3)
再对这三个维度分别进行 4，2，1 repeat
