If nothing about owning and riding bikes makes you at least a little scared, you’re probably delusional. I think it’s normal and natural to have fear. But at a certain point, while you may still be afraid to crash, you should not be afraid to ride. Real fear can be debilitating and will color your perception of situations — and that’s not good when you need your wits about you.
Personally, I gave up riding for 10 years and a good part of the reason was because I was afraid that I might get hurt and be unable to provide for my young family. I also had just moved to an island with very slick roads made from crushed coral and with tons of horrifically bad drivers. You might call it fear or sanity, but the feeling that a big crash was coming became overwhelming and it just wasn’t fun. So I took a break.
Don’t be freaked that you’re afraid. Skill and experience will likely overcome it. And if it doesn’t, find something else to do with your time and cash.