Remember that your bike makes power much higher in the rev band. If you are in too tall of a gear (4 instead of third or even second) then it’s tough to get going. Doesn’t sound like you’re doing that, but certainly stay (around) the middle of the rev range. Think of down shifting like this; imagine you will stay in the gear you just down shifted too. Then, as you slow more, shift to the next lowest gear you would normally ride at that speed, and so on…make sense?