Do you ever drop off of calls?

I HATE being in meetings that I'm not needed in, but I always have a conflict as to whether it's rude to drop off of meetings.