I believe you are referring to the novel [*Peter Graves: An Extraordinary Adventure*][1] by William Pène du Bois. 

*"A well-meaning but mischievous boy who encounters a gentlemanly and not-very-mad scientist named Houghton Furlong. Furlong is the inventor of an antigravity material named Furloy, and a Furloy-based invention called "the ball that bounces higher than the height from which you drop it." In an unfortunate accident with the latter invention, Peter destroys Houghton's house. Little of value is left in the wreckage except six balls of Furloy, each about the size of a golf ball, with an antigravity pull of 25 pounds-force (110 newtons) each. Peter commits himself to spending the summer with Houghton in an attempt to earn the $45,000 necessary to rebuild his house. The implied puzzle is: how can Peter and Houghton make use of the six Furloy balls to earn $45,000?"*



> While reading through Sci-fi pre-1985 children's(ish?) book series about a boy who lived next door to an old scientist, I was suddenly struck with a memory of having read a book within the last few years (I'll start trolling through my library history tonight after work) with a kind of similar premise. The inventor in question had some sort of pull or influence over the town such that they couldn't quite get rid of him. I want to say that he developed projects for the U.S. Military, although he was sort of a "gentleman inventor" who was doing it for the fun of it rather than someone who was truly trying to push the boundaries of human understanding. The two bits that stick out to me are
>
> He frequently causes incidents that require the fire department to show up, and he's got his own siren or klaxon for when things go wrong. 
>
> I don't remember if it was fires or gasses that were what kept happening, and I have a vague memory that the post office guy often got caught in the middle of incidents, to the point where the inventor treated him as a bit of a guinea pig.
He eventually gains a young boy as an assistant. 

Yep. He frequently causes disruptions to the town. In one case the neighborhood ladies association comes to visit.  He has a fireworks and exploding smoke bomb display to meet them. 


> We're introduced to said boy as he and a gang of kids are playing a "follow the leader" game where he's trying to winnow them out by doing progressively more dangerous or at least damaging to the clothes they're in. One which sticks out in my head was that they had to wade through a river, I think without removing their shoes, etc. The last challenge involves them going onto the grounds of this inventor, which I'm pretty involved traversing a fence. The young boy winds up the only one left when he encounters the inventor.

Match.  I believe the phrase was:  "In that moment two records came crashing down. Peter became the first leader to shake off all members in the follow the leader game. And became the first person to pay a single social call on Houghton Furlong."

> From there, I think I had to return the book, but the inventor decides to give the boy a job. I think he initially gives the boy some recent invention... I want to say they were either discs or glass bottles but they're on a belt, and there's something in the belt indicating that it was a military project. 

Not military. But the invention could have military applications along with the personal smoke screen pills the inventor had developed. 


> The book had the feel of an older book, maybe pre-computers. I'm almost certain it was set in a time before cell phones and personal computers, when the primary entertainment for kids involved romping through woods and creeks. I think the cover might have been yellow, a hardback, just the name of the book without a cover image? I'm pretty sure it was written for a younger crowd.

Yep.  Published 1950. 


  [1]: https://en.wikipedia.org/wiki/William_P%C3%A8ne_du_Bois