This is Clifford D. Simak's "A Death in the House" (text available here)
Old Mose Abrams was out hunting cows when he found the alien. He didn't know it was an alien, but it was alive and it was in a lot of trouble and Old Mose, despite everything the neighbors said about him, was not the kind of man who could bear to leave a sick thing out there in the woods.
Mose went to the cupboard and got the cigar box almost full of silver dollars and paid the doctor. The doctor put the dollars in his pocket, joshing Mose about his eccentricity.
But Mose was stubborn about his silver dollars. 'Paper money don't seem legal, somehow,' he declared. 'I like the feel of silver and the way it chinks. It's got authority.'
Farm life went on as usual, with the corn laid by and the haying started and out in the garden the strange plant kept on growing and now was taking shape. Old Mose couldn't believe his eyes when he saw the sort of shape it took and he spent long evening hours just standing in the garden, watching it and wondering if his loneliness were playing tricks on him.
The morning came when he found the plant standing at the door and waiting for him. He should have been surprised, of course, but he really wasn't, for he had lived with it, watching it of eventide, and although he had not dared admit it even to himself, he had known what it was.
For here was the creature he'd found in the woods, no longer sick and keening, no longer close to death, but full of life and youth.
Next morning, while Mose was fixing breakfast, he reached up in the cupboard to get the box of oatmeal and his hand struck the cigar box and it came crashing to the floor. It fell over on its side and the lid came open and the dollars went free-wheeling all around the kitchen.
Out of the corner of his eye, Mose saw the critter leaping quickly in pursuit of one of them. It snatched it up and turned to Mose, with the coin held between its fingers, and a sort of thrumming noise was coming out of the nest of worms on top of it.
It bent and scooped up more of them and cuddled them and danced a sort of jig, and Mose knew, with a sinking heart, that it had been silver the critter had been hunting.
So Mose got down on his hands and knees and helped the critter gather up all the dollars. They put them back into the cigar box and Mose picked up the box and gave it to the critter.
He remembered the object in his hand and lifted his still clenched fist in front of him. He opened his fingers and the little crystal ball lay there in his palm — and it was exactly like the one he'd found in the slitted flap in the body he had buried in the garden. Except that one had been dead and cloudy and this one had the living glow of a distant-burning fire.
Looking at it, he had the strange feeling of a happiness and comfort such as he had seldom known before, as if there were many people with him and all of them were friends.
He closed his hand upon it and the happiness stayed on — and it was all wrong, for there was not a single reason that he should be happy. The critter finally had left him and his money was all gone and he had no friends, but still he kept on feeling good.
Found by searching for short story alien grave "silver dollars"