激活函数
silu: 参考链接：https://pytorch.org/docs/stable/generated/torch.nn.SiLU.html
silu(x)=x∗σ(x),where σ(x) is the logistic sigmoid.
silu(x)=x*sigmoid(x)
$$ silu(x)=x*\frac{1}{1+exp(-x)} $$
silu: 参考链接：https://pytorch.org/docs/stable/generated/torch.nn.SiLU.html
silu(x)=x∗σ(x),where σ(x) is the logistic sigmoid.
silu(x)=x*sigmoid(x)
$$ silu(x)=x*\frac{1}{1+exp(-x)} $$