Perhaps "In the Bone" by Gordon R. Dickson from 1966?
This was published in a number of collections. Descriptions of the stories from each are listed below.
Points that match:
Trying to trace a short scifi story that appeared in print no later than the 1970s.
Check. Short story from 1966.
Premise is that an astronaut/pilot has crashed on a planet and lost most of his technology and memory.
Check. The protagonist is an explorer/astronaut whose powersuit is brought down crashing by the alien on the planet .
He does not remember being a pilot,
Check. Alien wipes his memory returning him to being a "beast".
... but he is aware that there's another, hostile, being on the planet using higher technology.
Check. The alien technology is much superior to the protagonists. Even as a "beast" he is aware of the alien.
Near the end of the story the pilot is triggered to remember who and what he is because while foraging he finds a small round object, ie his old helmet.
Partial check. In this story, the astronaut sees his old rusting powersuit equipment presumably including his old helmet, etc. But, it is when he sees his face in the river that he recognizes himself and starts coming out of his beast state.
He develops a plan to kill the other party and recover his technology.
Check. This particular plot point and how it works is somewhat similar to Frederic Brown's Arena.
Net, he finds an impenetrable barrier that kills any animal trying to get through. However, he finds that it stops at the top of the water on a small stream that goes under the barrier. He is able to get inside the barrier and successfully kills the alien gaining the technology.
At the end
He is then able to leave the planet.
Check. He learns how to pilot the alien ship back to orbit and returns to Earth.
In the Bone Collection
"In the Bone (1966) - 5/5 - Man technologically equipped to traverse the stars and explore new worlds finally meets another intelligence, who zaps him from the sky, disables his precious machinery and reverts the modern man back to his primal self to survive in the alien landscape in the shadow of the alien pyramid which struck him down. 22 pages "
Ancient My Enemy Collection
"In the Bone” Tied for the best story in the collection—Harry Brennan sets out to explore the galaxy in a fantastic mechanical contraption/body armor/spaceship/communication device. However, when he arrives on World 1242 a bizarre mechanized black pyramid object detaches him from his contraption and transforms him to an animalistic state. Filled with animal passions he is desperate to strike back at the pyramid with its animal catching devices and endless chambers. Dickson explores the traditional dichotomies man vs. machine, the primitive vs. the civilized, animalistic vs. sentient successfully. Recommended (reprinted in multiple collections)."