Zahara is a lovely place and there are lots of things to see and do in the region. It might be a bit chilly and windy in November though. In the imediate vicinity, don't miss the ruins of the Roman city Claudio Baelo in the coastal village of Bolonia. Tarifa, a little further afield is also quite charming. You may even consider hopping over to Morocco for the day (ferries from Tarifa and Algeciras). You should definitely spend a day in Cadiz, and try to see its archaeology museum. Another worthwhile trip would be to Jerez. Make sure you've got a designated driver who doesn't mind missing out on the sherry though.
Inland, I would recommend Arcos de la Frontera. From there you could explore the mountains and the wonderful white villages such as Grazalema, Zahara (a different one) and a must-see, Ronda.
Of course you'll want to see Sevilla, and you could easily spend more than a day there. If you have time, check out the ruins of the Roman city of Italica, about 3 miles out of town.
As you can see, there's lots to do, and you'll wish you'd come for longer. If you've any queries, get in touch.
Have fun