WTF is a Parameter?!?

WTF is a Parameter?!?
Demystifying the concept of a parameter in machine learning: what they are, how many parameters a model has, and what could possibly go wrong when learning them.