There’s enough room for a sweatshirt underneath.
I live in the Washington DC area, so we have similar climates. I wear the Fieldsheer in cool weather, with the liner in, and it works great. Last weekend, it dipped into the 50s in the evening, and riding home with the liner in and just a button-down shirt, I was comfortable.
In the fall and winter though, I switch to a leather jacket.