So I paid attention and noticed that I don't use the driver's door armrest while driving. So long as I keep my hands on the wheel, I don't use it. My 2015 doesn't have the seat-attached armrest, nor a center storage box.

The armrests are not a problem for me.