If you are forgetting the setup and getting a few details slightly off, you could well be talking about Heinlein's 'Coventry', published as part of Revolt in 2100. (It was written in the 40s, and collected in Revolt in 1953, but a lot of RAH was reprinted in the late 60s and early 70s due to the influence of 'Stranger in a Strange Land.')
The hero commits a social crime (punching someone in the nose) and is given the choice of being 'treated' to bring his way of thinking into compliance with society, or being exiled (with anything he wants to take with him) to a wild (and not very hospitable looking) place (separated from modern society) called 'Coventry.'
He chooses exile, and loads up a very slow moving vehicle and drives it in. Shortly after crossing, he's robbed of his vehicle and goods by highwaymen posing as Customs inspectors.
- and opened with a character driving an armoured, turtle-shaped vehicle
Well, it doesn't open with this, but the preceding part of the story is setup, and its with this part that the story seems to take off. The vehicle he is using to transport his possessions is a large, slow moving 'steel tortoise'. It's describe as "extremely rugged, easy to operate, and almost foolproof. It drew its power from six square yards of sunpower screens on its low curved roof. These drove a constant-load motor, or, when halted, replenished the storage battery against cloudy weather, or night travel. The bearings were 'everlasting', and every moving part, other than the caterpillar treads and the controls, were sealed up, secure from inexpert tinkering."
- across some desolate land.
He starts in the normal 'modern' world, and drives into what he perceives as a desolate area (even though it's not.. but to his modern eye, it looks that way.) As the force field opens: "The opening grew until it was twenty feet wide, then stopped. It framed a scene of rugged, barren hills. He took this in, and turned angrily on the legate. 'I've been tricked!' he exclaimed. 'That's not fit land to support a man.'"
- got stopped/held up a border crossing, check-point or similar, where his vehicle was inspected, with most of his gear (and possibly vehicle) being impounded or stolen from him
Yep. Soon after entering in, he's told to halt, and men claiming to be customs inspectors take all of his stuff. He's hauled off to a kangaroo court where he basically is fined everything he has, and then some, setting him up to be slave labor. He's tossed in jail, but helped to escape by a fellow prisoner.
- He then began to continue his journey on foot....
After helping him escape, his fellow prisoner tells him 'I guess this is the end of the line,' kid,' he told Dave. 'If you follow this street, you'll come to open country shortly. That's what you wanted, wasn't it?', and he walks off.