Members of five housing unions broke through security at the Social Housing Agency (AVS) headquarters and staged a sit-in. They called for a formal negotiating forum with the authorities to discuss eviction cases, debt and disputed housing situations.
Activists chanted slogans calling for secure housing and collective negotiations as police arrived at the building.
The protesters were eventually removed from the offices and their identities were checked by police as they left.
The action follows widespread protests triggered by the eviction of an 87-year-old woman from her Madrid home last week.