[Thought] A lower learn rate might cause underfitting

This is a personal thought I had today while reviewing about overfitting and underfitting. This opinion might be wrong and might be changed while I learn and review over time. If you have any other thoughts or corrections, please write a comment and I will try to apply it to this page with the username.

 

When testing models, choosing the learn rate was not an important factor. The only caution I had was ‘When using a high learn rate the loss might increase exponentially‘.

 

While reviewing and writing the page for the ‘Bias and Variance tradeoff’. I thought about how learn rate might affect underfitting or overfitting.

 

Having a low learn rate might produce smaller changes in the model, trapping the model in a local minimum. This might be a reason for underfitting if the model gets trapped during the beginning of training. So my thought is that having a too high or too low learn rate might produce underfitting.