You’ve identified the problems pretty well – cold, stored in the open, no battery tender, ran infrequently. When you do start the bike, do you run it long enough, at high enough rpm, for the battery to charge? It’s generally true that a fully drained battery would suffer some degree of permanent damage, but it seems yours was just low.
The only thing I can think of is to remove the battery from the bike and store the battery indoors. Even better if you could have the battery hooked up to a battery tender indoors.