As Mat Cauthon's answer points out, The Houses of Healing is a less intense chapter that comes after a major battle and not long before another battle. Writers usually try not to wear out their readers with constant action and tension.
I think we can call it "comic relief" in the traditional sense (as in Shakespeare's comedies), but these days we would probably say it is "light relief".
There a few comic moments
Aragorn putting the herb-master in his place
Thereupon the herb-master entered. ‘Your lordship asked for kingsfoil, as the rustics name it,’ he said; ‘or athelas in the noble tongue, or to those who know somewhat of the Valinorean...’
‘I do so,’ said Aragorn, ‘and I care not whether you say now asëa aranion or kingsfoil, so long as you have some.’
The Lord of the Rings Book Five, Chapter 8: The Houses of Healing
Pages 864-5 (Houghton Mifflin Harcourt; Single Volume 50th Anniversary Edition)
Aragorn teasing Merry
‘Master Meriadoc,’ said Aragorn, ‘if you think that I have passed through the mountains and the realm of Gondor with fire and sword to bring herbs to a careless soldier who throws away his gear, you are mistaken. If your pack has not been found, then you must send for the herb-master of this House. And he will tell you that he did not know that the herb you desire had any virtues, but that it is called westmansweed by the vulgar, and galenas by the noble, and other names in other tongues more learned, and after adding a few half-forgotten rhymes that he does not understand, he will regretfully inform you that there is none in the House, and he will leave you to reflect on the history of tongues.
The Lord of the Rings Book Five, Chapter 8: The Houses of Healing
Pages 869-70 (Houghton Mifflin Harcourt; Single Volume 50th Anniversary Edition)
The greater part of the chapter advances the plot.
Although the chapter does provide some relief from the intensity of the battle, it also has a serious purpose
Establishing Aragorn's right to the throne of Gondor
Ioreth (one of the "comic" characters) is the one who says
Would that there were kings in Gondor, as there were once upon a time, they say! For it is said in old lore: The hands of the king are the hands of a healer. And so the rightful king could ever be known.’
The Lord of the Rings Book Five, Chapter 8: The Houses of Healing
Page 860 (Houghton Mifflin Harcourt; Single Volume 50th Anniversary Edition)
This leads Gandalf to call for Aragorn who proceeds to heal Faramir, Éowyn and Merry.
When Faramir awakens, he acknowledges Aragorn as the King
Suddenly Faramir stirred, and he opened his eyes, and he looked on Aragorn who bent over him; and a light of knowledge and love was kindled in his eyes, and he spoke softly. ‘My lord, you called me. I come. What does the king command?’
The Lord of the Rings Book Five, Chapter 8: The Houses of Healing
Page 866 (Houghton Mifflin Harcourt; Single Volume 50th Anniversary Edition)
Ioreth isn't going to keep this to herself
As he followed Gandalf and shut the door Pippin heard Ioreth exclaim:
‘King! Did you hear that? What did I say? The hands of a healer, I said.’ And soon the word had gone out from the House that the king was indeed come among them, and after war he brought healing; and the news ran through the City.
The Lord of the Rings Book Five, Chapter 8: The Houses of Healing
Page 866 (Houghton Mifflin Harcourt; Single Volume 50th Anniversary Edition)
Aragorn and Éowyn
Gandalf explains to Éomer how Éowyn came to despair.
‘Think you that Wormtongue had poison only for Théoden’s ears? Dotard! What is the house of Eorl but a thatched barn where brigands drink in the reek, and their brats roll on the floor among their dogs? Have you not heard those words before? Saruman spoke them, the teacher of Wormtongue. Though I do not doubt that Wormtongue at home wrapped their meaning in terms more cunning. My lord, if your sister’s love for you, and her will still bent to her duty, had not restrained her lips, you might have heard even such things as these escape them. But who knows what she spoke to the darkness, alone, in the bitter watches of the night, when all her life seemed shrinking, and the walls of her bower closing in about her, a hutch to trammel some wild thing in?’
The Lord of the Rings Book Five, Chapter 8: The Houses of Healing
Page 867 (Houghton Mifflin Harcourt; Single Volume 50th Anniversary Edition)
Tolkien doesn't tell us much about Aragorn's love life (outside of the the Tale of Aragorn and Arwen in Appendix A), and at this point the reader may not understand why Aragorn has not responded to Éowyn's advances. In this chapter Tolkien at least gives us a clue.
But Aragorn said: ‘I saw also what you saw, Éomer. Few other griefs amid the ill chances of this world have more bitterness and shame for a man’s heart than to behold the love of a lady so fair and brave that cannot be returned.
The Lord of the Rings Book Five, Chapter 8: The Houses of Healing
Page 867 (Houghton Mifflin Harcourt; Single Volume 50th Anniversary Edition)