Harry is in Gryffindor house. If he wanted to go and visit Malfoy in Slytherin house, could he do it?
No, Hogwarts students can only enter their own House Common Room.
The different House Common Rooms all have some form of security at their entrances to keep out non-residents. The Slytherin Common Room is protected by a password. Harry and Ron see this in action when they use the Polyjuice Potion to impersonate Crabbe and Goyle.
Malfoy paused by a stretch of bare, damp stone wall. “What’s the new password again?” he said to Harry.
“Er” said Harry.
“Oh, yeah - pure-blood!” said Malfoy, not listening, and a stone door concealed in the wall slid open.
(Chamber of Secrets, Chapter 12, The Polyjuice Potion).
The fact that there was a new password also demonstrates that the password changes on a fairly regular basis. Despite finding out the password by illicit means, Harry and Ron wouldn't have been able to reenter the Common Room if they'd tried to do so a few months hence as there would be a new password in place.
Gryffindor's Common Room is also protected by a password. Ravenclaw's Common Room uses logic puzzles to sift out non-Ravenclaws.
“What? Isn’t there just a password?”
“Oh no, you’ve got to answer a question,” said Luna.
“What if you get it wrong?”
“Well, you have to wait for somebody who gets it right,” said Luna. “That way you learn, you see?”
(Deathly Hallows, Chapter 29, The Lost Diadem).
Meanwhile, Hufflepuff uses a system of tapping on barrels that apparently has not been duped at any point in the history of Hogwarts.
What else do you need to know? Oh yes, the entrance to the common room is concealed in a stack of large barrels in a nook on the right hand side of the kitchen corridor. Tap the barrel two from the bottom, middle of the second row, in the rhythm of ‘Helga Hufflepuff’, and the lid will swing open. We are the only house at Hogwarts that also has a repelling device for would-be intruders. If the wrong lid is tapped, or if the rhythm of the tapping is wrong, the illegal entrant is doused in vinegar.
You will hear other houses boast of their security arrangements, but it so happens that in more than a thousand years, the Hufflepuff common room and dormitories have never been seen by outsiders. Like badgers, we know exactly how to lie low – and how to defend ourselves.
Once you’ve opened the barrel, crawl inside and along the passageway behind it, and you will emerge into the cosiest common room of them all.
(Welcome to Hufflepuff message on old Pottermore).
This was addressed in the Harry Potter and the Philosopher's Stone GBA video game. Students from other houses aren't allowed to enter their counterpart's common rooms.
"Why are they called Common rooms when only people from a certain house can go into them? Doesn't that make them uncommon rooms?" - Ravenclaw Student (with a bezoar in his pocket)