How do you make a cover song feel like your band without wrecking it?

Cover songs can either sound like a weak copy or like a version with its own personality. The tricky bit is changing enough to make it yours without losing what people love about the original.

What do you change first when arranging a cover: key, groove, tempo, guitar tone, vocal phrasing, structure, or dynamics? And what parts do you avoid messing with?