The rowers had been free men of Gondor and the surrounding areas who had been enslaved by the Corsairs. They did provide the manpower, but at that point they were no longer slaves. Gimli narrates the events:
That night we rested while others laboured. For there were many captives set free, and many slaves released who had been folk of Gondor taken in raids; and soon also there was a great gathering of men out of Lebennin and the Ethir, and Angbor of Lamedon came up with all the horsemen that he could muster. Now that the fear of the Dead was removed they came to aid us and to look on the Heir of Isildur; for the rumour of that name had run like fire in the dark.
And that is near the end of our tale. For during that evening and night many ships were made ready and manned; and in the morning the fleet set forth. Long past it now seems, yet it was but the morn of the day ere yesterday, the sixth since we rode from Dunharrow. But still Aragorn was driven by fear that time was too short
'"It is forty leagues and two from Pelargir to the landings at the Harlond," he said. "Yet to the Harlond we must come tomorrow or fail utterly."
The oars were now wielded by free men, and manfully they laboured...
(The Lord of the Rings, Book V, Chapter 9, "The Last Debate"; emphasis added)