John Rennie actually said this in his answer already, but only as a sidenote.
The crucial distinction to think about, IMO, is that between a vector space (or an affine space) and a manifold. This is actually pure mathematics, but I dare to give an overview here anyway.
A vector space is the kind of mathematical space that we're familiar with; in such a space "3-dimensional" means that you can pick any point and describe its location very simply with just 3 numbers: you need some conventional basis of three orthogonal1 vectors, and then you just say "go 4' in direction ex , then 7' in direction ey and finally 2' in direction ez ." Important: this description is unique, i.e. once you've fixed your basis there's no different alternative way to describe that point. That means that, to get from one point to another one, you always have to travel that distance. There are no short-cuts, in the sense that you can always find the shortest route by just walking as staighly forwards as possible, namely in a straight line.
A manifold is more general. A manifold is locally a vector space. The usual example is the earth's surface: if you're just interested in a small area, you can easily make a map of it, which is a (bounded) 2-dimensional vector space. Still, the earth surface as a whole is not a vector space, but a 2-sphere. That's a very simple manifold: its fundamental group is trivial, which means that there is still a unique shortest way that can be found by going as straightly as possible, i.e. to be found by tightening a string between the two points on a globe. However there exist more complicated manifolds, for example the 2-torus; think of a doughnut's surface. On such a manifold, there are multiple topologically different ways to go, and it's not possible to, and it's not possible to determine the shortest route by pulling a single string. Now it might be possible that one knew only about one of the possible routes, and be utterly surprised that there was a shorter way all the time: a shortcut.
What does this have to do with dimensions? The thing is, we know pretty certain that "our normal" 3-dimensional space has no such notrivial topological features, so if there were to be shortcuts they'd need to be embedded in a higher-dimensional space that we can't observe right now. That's the "extra-dimension".
1Actually, linearly independent is sufficient.