A lot of market stalls will give you herbs for free with your purchase (bay leaves and parsley in particular). Most flower shops sell little herb plants, though I've found that the climate here is so dry that it's hard to keep them alive (except for rosemary, which grows like a weed).