This is the timeline for the Tesseract and its use by humans:
c. 1942: Red Skull recovers the Tesseract from Norway. Red Skull's HYDRA faction begins developing weapons using it as a power source.
c. 1943-45: the main conflict between Captain America and HYDRA. The Tesseract is lost in 1945 with Captain America. Howard Stark recovers the Tesseract.
What happens between 1945 and the late 1980s with the Tesseract is currently unclear. Howard Stark, worried about the dangers it might represent, might have kept it locked away within SHIELD until Lawson/Mar-vell took it to power the Light Speed Engine, whether it was given to her or she secretly stole it and Project Pegasus didn't know it was the power source is unknown. Or he might have secreted it away keeping it from everyone and gave it to her. Or she might have stolen it. Who knows at this point. Whatever the case, as of 1989 it wasn't available for human use any more until sometime after 1995 when Goose harked it up on Fury's desk.
Now, again this is hypothetical, but given the secrecy involved, Fury might not have been aware that the item he now had was the same one that powered HYDRA weapons in the 1940s. Stark was killed in 1991, and Peggy Carter, as well as everyone else who'd been around and knew firsthand the Tesseract powered weapons and might have stayed with SHIELD, should have been retired or dead by 1995. So, as far as he knew, the only thing the device did was provide power for a propulsion system that might or might not be useful for weapons. And, in any event, what happened in 1995 wouldn't have necessarily driven the need for radical weapons development: the Skrulls and Kree had left, and based on Fury's first-hand knowledge, they didn't need to develop any special weapons to deal with them on an individual level: he'd seen a Skrull killed by purely mundane means (the one pretending to be Coulson) and had no particular reason to believe the Kree would be any harder to deal with. Further, the Skrulls had left and the Kree only came to Earth because of them, and based on Carol's knowledge, they didn't give a crap about Earth otherwise.
(Incidentally, Coulson never saw a Kree aside from Vers, so he had no reason to know they were blue; or it might have been erased from his memory as a result of Project TAHITI, which is why he didn't recognize the Kree body years later on Agents of SHIELD.)
So the only threat that Fury knew normal portable weapons couldn't deal with was Carol Danvers herself, and it's quite clear for the admiration he has for her by the end of the film he doesn't consider her a threat.
So, he's got a power source that he might not know could be used for weapons (because he doesn't know what it is), and nor particular reason to develop more advanced weapons.
Then 2011 in New Mexico happens (Thor) and he realizes how hilariously outgunned humanity really is. Now there's someone normal weapons can't deal with.
Again, hypothetical: he goes scrambling to see might work, and looks at those weapons SHIELD inherited from the SSR that were captured from HYDRA, and finally makes the connection between Goose's little gift and the power source from the 1940s, and pulls Selwig in (Thor credit stinger) to start making advanced weapons again.