Hard to pay - Is it just me?

Is it just me?, but although I normally get prompt and efficient service in the bars here (Madrid), I always find it hard/time-consuming to get the bill and pay up.
Perhaps a Spanish person could enlighten me as to why this would be the case?

Yes, that's true very often, try to order the bill with the last thing you order (if you know that will be the last order).
For example:
'un café solo y la cuenta por favor', they'll bring you both together.

That may be considered one of the nice things about Spain. The 'wait staff' doesn't want you to get the impression that you are being rushed out the door when you finish your meal (or drink & tapas). Sometimes it can be irritating.
If I can't get my waiter's attention when I'm ready to leave, I'll just get ready to leave. The waiter (or barman) usually will appear with the bill. If I can get their attention, I make a 'writing' gesture to indicate that I want my bill. If all else fails, I go to the register at the bar and say: "Cuanto va?"

At outdoor *terrazas* in Madrid, I usually have to flag down the waitperson as s/he is going around to the other tables. They assume you're following local custom, and may be staying for an hour or two.

At restaurants in much of Spain, you can indicate you're done by placing the knife and fork together on your plate at the "4:20" mark (i.e., if the plate were a clock face, both are on the number 4). Be careful not to give into the temptation to set down your knife and fork opposite each other at the "9:15" mark -- that means you hated the food.

Do what I do...look around, squirm in your seat uncomfortably, take out your purse/wallet and start going through it, look around again, give up and sip on an empty glass, look around again, look quizzically at your companion, watch the TV over the bar as if you understand the soccer match that's on, look around again, make eye contact with the waiter and smile, fiddle with your wallet again, stand up, sit down, sip empty glass once more, stand up and put on jacket, sit down and stare at TV again, smile at OTHER waiter, then FINALLY do what Eddie suggests and gesture to the waiter/waitress with a little "writing" motion.

Pay, leave, and spend the next couple of minutes thinking about how you would do it differently next time. Worry about seeming like a pushy American.

Repeat, endlessly, on every trip to Spain.

and it's not just Spain... And every time I go through that I think the waiter thinks that I am a crazy American who is "always in a hurry".

I usually do like Miguelito, it works most times. Else, I do as Eddie, signs, standing up (if I am fed up) or going to the cashier and asking "¿Cuanto es?".

I am so delighted to hear that even Spanish people have trouble paying when it's time to leave a bar or restaurant! I had thought I just hadn't learned some magic code or signal yet!