简要回顾 PyTorch 中关于 Tensor 形状变换的 API（如 view, reshape, unsqueeze, squeeze）以及 Einops 库中 rearrange, reduce, repeat 的用法。

This blog revisits the underlying principles of the Vision Transformer (ViT) and proposes explorations on extending the Transformer architecture to other modalities. For instance, in tasks such as "Weight2Weight," prior approaches have often simply flattened weights into one-dimensional tensors, without leveraging positional encodings.

1. 梯度范数 (Gradient Norm)在深度学习中,梯度是一个向量,表示损失函数相对于模型参数的偏导数。对于具有 n 个参数的模型,梯度可以表示为: $$\nabla L = (\frac{\partial L}{\par