First of all, Coach or not, that's one brand that is not known over there, so if you're thinking someone will snatch it because of that, I'd say no. A purse is a purse, and most likely has money, that's what they're thinking and pretty much it. The fact that its a purse that hugs close to you, that's a good thing. Most of my purses are that kind too.
Its hard for anyone to say take a purse or not with you... if you wear a fanny pack, you will scream tourist and won't be any safer! If you're going to get mugged, it will happen, like you said, the best you can do is to be aware.
I only had my purse stolen once, and it was pretty stupid- a friend and I were taking turns holding the small bag as I told her I'd hold her stuff also- we were in a bar at the beach where another close friend worked and its was mostly empty and although its a nice bag easily and comfortably held in one hand, she didn't want to hold it and set it up on the bar. Shortly thereafter, the bar started to fill up and I went to get the purse because I didn't think it was safe, and sure enough, the purse was gone!
I was lucky- we did end up finding it, with my ID's but the money, lighter and my favorite lipstick were gone!(What someone would want with a used lipstick is beyond me!) Oh well, nothing that couldn't be replaced! I am a bit paranoid about the passport(too much of a pain if it gets lost/stolen), so no, I don't carry it around with me, I'm comfortable enough there that I really don't feel there's a need- I had my US drivers license and a student ID in there, both of which were recuperated. I also don't go out dancing with any Credit cards, I carry the money I've got with me, and if one of my guy friends is with me, I usually give them some to hold for me- just to have some in a different place.
The thieves are getting pretty gutsy- if you don't carry a purse, they're going to imagine that you're carrying money on you SOMEWHERE! So, either carry a purse and don't put everything in there (i've done that to, in pockets, bra, even my shoe!) so if they want to take something- they'll go for the obvious (so if you don't want to loose your Coach- take a less expensive purse) or risk being held up at knife point and made to prove you don't have any cash on you or even have them check for themselves! Your call. Just do what you think makes best sense.
_________________________
Madrid!