I highly recommend Peñafiel as well. Bone up on your history there before going, though. This is an extremely unique castle...it looks like a ship lodged on a hill. Depending on the time of year, you can actually drive up to the front gate (even though the bus is free). There is a corresponding hill a ways away which offer a great staging area from which to take beautiful pictures. If you go in May, the entire countryside around Peñafiel explodes red with poppys (or is it poppies
?).
If you're going around Segovia, there is a little itinerary that I took my father on which took us to five castles in a day. It included Coca and Pedroza (if my memory serves me), along with the ones to follow:
Cuéllar, which lies at the midpoint of Segovia and Valladolid, is worth a stop. It has been renovated and is now an instituto. You can usually catch a profe there who is willing to answer questions. The old part of town is cutesy, though small.
If you find yourself heading to Valladolid from Segovia, after stopping in Cuéllar, go to Portillo, which has an excellent old town arch-gate, and an odd-looking castle which I believe is a fine arts school at present. Top it off with a visit to Simancas' castle at the outskirts of Valladolid (home of the national archives) and you've seen some of the finest!
I guess that's why it's called Castilla...