With a three year old in tow , I would stick to the centre , most of which is perfectly safe to be in . Certain areas such as Lavapiés have had some bad press ( racial tensions ) , but most areas are fine . The 'Barrio' of Salamanca where I lived is very conservative , with good Metro links but not right in the middle of the city BUZZ areas . Chueca is the gay area and therefore the party area .. All areas around Chueca , Gran Vía , Sol will be noisy well into the early hours .Great fun unless you want to sleep . Some of the areas around Retiro Park are nice , even up towards Atocha .. Sort of medium busy .. Madrid's public transport system is vast and very efficient so you'll have no problem getting around . A nice quiet area to stay within easy reach of all the major areas is around the Prado Museum , though the Metro stations are a little spaced out . I stayed in Calle Cervantes many years ago , very near to the Congress building .. just a 15 minute walk to Sol ( max ) .. A very relaxed area within the central part of Madrid ( Also home to The Ritz hotel ) . The main Metro stations would be Atocha , Sevilla and Sol .