After the events of "A Good Man Goes to War", the Doctor comes to realize that the choices he has made and the life he has lived is making him a target, and that because of his compassion for his companions, they become targets as well. It's after this point that he starts to distance himself from Amy and Rory. Not because he doesn't care about them anymore, but because he cares too much.
We've seen the Doctor's previous companions in the new series become very attached to him, and this makes them liabilities. He's also been previously admonished for turning those around him into weapons, partly because of their trust and faith in him. It's a sobering wake-up call.
He doesn't want his enemies coming at him through his friends again, so he holds them at arms' length in order to protect them. That means letting them live their own lives, and only occasionally popping round to whisk them off on an adventure.