The official Marvel page for Bruce mentions his affiliation as:
It’s also worth noting that Bruce and the Hulk are one and the same, you can’t have one without the other and so Bruce must be an Avenger by default just because of that.
We then have the fact that Bruce in the MCU shows up at multiple “Avengers meetings” and if it was only the Hulk who was a member this wouldn’t be the case. Bruce is a brilliant scientist, similar to Tony Stark but works in a different field and so even on his own he is a massive asset.
Also as @NKCampbell notes in a comment in Thor: Ragnarok the Quinjet notes Bruce as the ”Strongest Avenger” when he is Bruce and gives the password ”Banner”.
QUINJET (V.O.): Voice Activation required.
QUINJET (V.O.): Welccome, Strongest Avenger.
In The Avengers when Natasha brings Bruce in she tells him Fury is only after Bruce not the Hulk which suggests they are recruiting Bruce and the Hulk to the Avengers.
NATASHA: Nick Fury seems to trust you. But now I need you to come in.
BANNER: What does Fury want me to do? Swallow it?
NATASHA: Well, he wants you to find it. It's been taken. It emits a gamma signature that's too weak for us to trace. There's no one that knows gamma radiation like you do. If there was, that's where I'd be.
BANNER: So Fury isn't after the monster?
NATASHA: Not that he's told me.