It seems like the Doctor ends up in places that need him most. Does the Doctor choose where the TARDIS goes or does the TARDIS itself? He can "see" the timeline (not sure of the details on that) but at times he seems surprised where he ends up.
There are definitely times when The Doctor has control of the Tardis and it goes exactly where he wants it to. For instance, in The End of the World he chooses to go to the day when the sun will expand and destroy Earth, and arrives there exactly as planned.
However, the Tardis does go to places that the Doctor did not intend, as you pointed out. In The Doctor's Wife, there's this great exchange:
The Doctor: You didn't always take me where I wanted to go.
Idris (inhabited by the Tardis): No, but I always took you where you needed to go.
The Doctor: You did!
So the Doctor chooses where the Tardis goes, unless the Tardis decides that he needs to be somewhere else.
In Old Who, the First and Second Doctors had little to know control over where/when the TARDIS would show up. As stated in "The Day of the Doctor", the navigation system was a bit messed up. There were also some indicatations that The Doctor didn't know how to control it exactly.
After his capture and exile by the Time Lords, The Doctor couldn't fly the TARDIS at all, at first. During this time, the TARDIS had been purposely sabotaged by the Time Lords and The Third Doctor's memories were blocked so that he couldn’t fix it or work around the sabotage. He did try, however, which is why the console was outside the TARDIS in Inferno. At times, the Time Lords took control and sent him where they wanted a job done (e.g. Peladon). After The Three Doctors, he received a new dematerialization circuit and the memory block was freed. From that point, he could travel as he wished, but still used UNIT as a base. He had a lot more control over where/when the TARDIS went, but still had trouble getting to Metabelius 3. More than likely, the Time Lords repaired the damage that prevented the first two Doctors from controlling the TARDIS' travel.
The Fourth Doctor then fine tuned his piloting skills to where he could move the TARDIS short distances in space without travelling through time. For example, he could move it to another location in the same city. Hoever, the Time Lords still showed the ability to take control of it from time to time. After "The Key To Time", he installed a randomizer so that it would be harder for the Black Guardian to follow him and no longer could control his destination. He later disabled it.
As others have stated, the TARDIS will still go off course when it needs to so later Doctors still don't have 100% control of where/when they end up. Examples in the current series are returning Rose a 12 months after they left instead of 12 hours and returning to pick up Amelia (Amy) Pond 12 years later and then another 2 years.
Sometimes, but a lot of the time, the TARDIS just randomly goes to a location, which is how a lot of the episodes start off.
The TARDIS is controlled by the doctor but other people can take control of it even when they don't have access to the TARDIS.This was demonstrated in the episode "The Doctors Daughter"