I've never seen an armrest in person yet, most of the ones I've seen on other vehicles usually have a zipper or something at the back tucked inside. Just unzip it and there's a bolt. If you do take it off you'll be left with a hole in the side of the seat.
Or if that doesn't work break out the Sawzall and cover the hole with duct tape.