Over-parameterisation
Created: 17 Nov 2022, 05:16 PM | Modified: =dateformat(this.file.mtime,"dd MMM yyyy, hh:mm a")
Tags: knowledge, MTL
[From MTL Problem Analysis Report:]{.underline}
Regularisation methods need to be carefully designed to avoid over-parameterised network such as AdaShare. <https://arxiv.org/abs/1911.12423>
- Why is over-parameterisation bad here?
[From MTL research direction analysis report:]{.underline}
The study aims to design a deep MTL method using over-parameterised convolutional layers to improve model expressivity and to learn task-specific variations. In a broader perspective, this study falls into both the MTL architecture design and model compression domains.
- Why is over-parameterisation good here?
- Why do you do over-parameterisation for training and not test? Why does that help?
What does over-parameterisation mean?
- https://towardsdatascience.com/overparameterized-but-generalized-neural-network-420fe646c54c
- https://www.lesswrong.com/posts/GfEmhoyS6ztk7GQAp/how-to-think-about-overparameterized-models
