This is Hour of the Horde by Gordon R. Dickson. It was first published (probably in abridged form) in Venture Science Fiction Magazine, May 1969; this version is available at the Internet Archive. The excerpts below are from the magazine version.
I think the man was one-armed or had one damaged/withered arm. He is an artist when we meet him.
It had happened again. That primitive, unconquerable power in him that he could not seem to deny, had reached out once more, savagely down the muscles of his good right arm and hand, to take over his painting.
Exhausted, Miles Vander threw the number four brush he held, now bloodily tipped with alarazin crimson, back into the pint fruit jar of muddy turpentine holding the other long, yellow-handled brushes. A feeling of dull exhaustion and frustration dropped on him like the doubled folds of some heavy blanket.
All at once he was aware again of his own starved-looking body, his bent shoulders, his uselessly hanging left arm that polio had crippled six years ago. The paralyzed hand was now tucked into his left pants' pocket, out of sight; and the loose sleeve of his white shirt, billowing about the wasted arm in the late sunlight of the warm spring afternoon, disguised for the moment its unnatural thinness.
He's picked up by aliens to represent earth in a galactic armada.
"Our visitors have told us," [the President] said, "that defense of the galaxy is a common responsibility. For our world to join in that defense, is therefore a duty. What they require from us, however, is a contribution of a highly specialized nature." His voice hesitated, and then went on, more strongly. "They tell us that the weapons with which our galaxy's defensive force will meet the Horde are beyond the understanding of our science, here on Earth. They tell us, however, that they are part physical, part non-physical in nature. The number of fighting individuals we can contribute to our galaxy's defense is therefore limited by our relatively primitive state of awareness as far as these non-physical forces are concerned. We can send one man, only. This one individual—this one man, who is best suited to be our representative by natural talent and abilities—has already been selected by our visitors."
Of course Miles Vander, the painter with one good arm, is the chosen representative of Earth.
Fight is to protect against invaders from another galaxy.
". . . These gentlemen," went on the presidential voice slowly, "have explained to the representatives of the nations of our world that our galaxy, that galaxy of millions upon millions of stars, of which our sun is a minor star out near the edge—" The figures of the two men disappeared and were replaced by what looked like a glowing spiral of dust floating against a black background, "will shortly be facing attack by a roving, intergalactic race which periodically preys upon those island universes like our galaxy which dot that intergalactic space.
"Their civilization, which represents many worlds in many solar systems in toward the center of the galaxy, has taken the lead in forming a defensive military force which will attempt to meet these predators at the edge of our galaxy, and turn them aside from their purpose. They inform us, that if the predators are not turned aside,m over ninety percent of the life on he inhabited worlds of our galaxy will be captured and literally processed for food to feed this nomadic and rapacious civilization. Indeed, it is the constant need to search for sustenance for their overwhelming numbers, that keeps them always on the move between and through the galaxies, generations succeeding generation in rapacious conquest."
Put in a ship with other aliens only there so a representative of their species can be said to be present - but not expected to actually contribute much to the coming fight.
"I was told our ship isn't ever going to fight the Silver Horde!" said Miles. "Is that true?"
"Quite true," answered the Center Alien.
"Then why bring me out here in the first place?" demanded Miles. "Why bring any of us—any of us on the Fighting Rowboat?" . . .
"It's true that individually, and even collectively, you add little directly to out general Battle Line strength," answered the Center Alien. "All of you together amount to less than one of the least of my own people, in this respect. But there is more than the direct addition of strength to be considered, Besides his own, personal powers, which vary widely from individual to individual, each one in the Battle Line has a function in which all are equal. That is to act as a resonator, or amplifier, of the group strength, and as a channel through which that strength may be directed at the enemy. There is what you might call a feedback effect—from the group to the individual and back again—where the psychic force is concerned."
Somehow or other, their ship and crew become integral to winning the war.
"Then you remember that the Center Aliens didn't have anything like what I had, in themselves. If they had, we'd have felt it. More than that, they'd never have needed to run from the Horde in the first place; if their naturally much greater psychic strength could be multiplied like ours did, by their going into overdrive."
"Unmultiplied, their strength was enough—once they did come back and start fighting," said Luhon.
"Yes, once they came back!" said Miles. But the point is, they didn't come. —Not until after we, with no hope, just instinct, had attacked the Horde and changed the battle odds for them. The odds meant everything to them—nothing else did."