If there's a possibility that you're misremembering a film as being a book (it's happened to me before), then you could be looking for the cult-classic film Brainstorm. I originally assumed you'd read a novelization of this film, but I've looked in all the places I'd expect to be able to find one and can't find any hint that one was ever published, so this seems unlikely.
Specific similarities to the story you're asking about are:
back in late 70s early 80s
Brainstorm was released in 1983
a scientist
The story focusses on a research group, headed by Michael Brace (played by Christopher Walken).
invents a machine that can record [and] playback [memories]
Brace and his group are developing a device that records all sensations and can replay them as if they are happening live at a later date. This gives an impression that is somewhat like a memory only more vivid[*].
and influence other people
An unexpected side-effect of the recording is that it also records emotions and replays those too. This leads to people being changed, e.g. Brace and his estranged wife reunite because they are now able to understand each others' emotions.
at the end the mafia or similar tried to get it
Not the mafia, but a sinister military dark-ops group, who want to use it for brainwashing and torture.
but he uses it to influence and beat them
Here the plot diverges from what you remember slightly -- he actually breaks into the research lab and destroys it in order to avoid it being misused.
[*]: fun fact - the film was written as a vehicle to demonstrate a technique devised by its director, special effects guru Douglas Trumbull (best known for 2001 A Space Odyssey), to provide a "more-vivid than real life" feel to the playback scenes, but it would have involved using special projection equipment and the studio decided not to release the film with it in the end...