...I'll save the day by proxy. Do NOT attempt both an El Escorial and Segovia excursion on the same day. You would cheat both cities, as either could hold you for a minimum full day (Segovia a little longer).
The Escorial monastery is an incredible treasure, dubbed the Eighth Wonder of the World at the time of its completion, it will occupy and fascinate you all day. The little pueblo in which it is situated is just that...a little pueblo. Most will do a Valle de los Caídos/El Escorial ½ and ½.
Segovia is worth two-three days to many, but most only spend a day there. There's plenty to fill your time, but the tourist trifecta is usually the acueducto, the cathedral (a gem!) and the alcázar. Loading up on cochinillo at the end of the day just puts the icing on the proverbial cake.
I suggest a night in Segovia, as the walls of the city, at the point where the acueducto enters the old part, are lit up at night, and provide a spectacularly singular sight. I've stayed at the Hotel Acueducto just up the road and both times have had street-facing rooms which provide a view of the city walls and acueducto from the veranda that make the price seem like a steal (it's not too expensive anyway...).
You can get to either place relatively easily. I suggest the train to El Escorial, and the bus to Segovia, although the cercanías (local-regional trains) seem to get to Segovia now without a hitch. Check
RENFE for exact timetables!