Isaac Asimov was famous (in my mind) for writing detective stories that happened to be set in space. I am also a fan of Joss Whedon's Firefly series for separating the science fiction setting from the genre. Why are sci-fi stories classified by where the action takes place, whereas detective and romance stories are classified by the type of story that takes place?
I think that sci-fi can be used to describe different things. It can be used to describe setting (for instance, the movie "Alien") or it can be used to describe genre, where some sci-fi concept is fundamental to telling the story, regardless of the setting (for instance, the movie "Minority Report"). The use of the phrase "sci-fi" in these two examples means different things to me.
Sci-fi is a broad category of fiction where some of the fictional ideas that are presented have some kind of scientific basis. It is a genre of fiction, just as Romance and detective novels are genres of fiction. The only difference is science fiction subdivides fiction in a different manner than Romance or detective novels.
I think it's simple to consider a novel to be both a Detective (or whatever) novel and a Sci-Fi novel. It's just that the fact that a work is Sci-Fi distinguishes it more in the public's eye than some other genre.
In order for a story to take place in a "sci-fi setting" (space, other planets, etc), there generally needs to be other elements of science in the story as well (otherwise there's little point in choosing that setting).
For example, Firefly is not just a western set in space. The series examines the impact of scientific changes (e.g. the societal impact of human diaspora to other planets, the moral implications of genetic manipulation, how language is likely to change when humans are spread over multiple planets, the economic reality of limited-resource space travel) as well as having space ships and planets with multiple moons. Most of the science falls under the social sciences (still science!), rather than physical sciences.
Asimov's stories are likewise not just detective stories set in space. The setting allows examination of scientific questions about psychohistory (mathematical physics), psychic powers, the rise and fall of cultures, the ethics/morality of human/robotic relations, and so forth.
If you took a story that is not considered science-fiction, e.g. Wuthering Heights, and changed the setting to another planet, but did not change anything else, would that be then science-fiction? My answer would be that the story would not be published, because it doesn't make any sense for the setting to be another planet and have that unrelated to the story. If you did change it, so that the story made use of the setting, then it is science-fiction, but that's no longer only because of the setting.
Remember that genre are not mutually exclusive. A story might be a romance, or a comedy, or a romantic comedy, just like a story might be a detective story, science-fiction, or a scifi detective story.
Among his 300+ books, Asimov wrote several that were "only' detective/mystery stories and not SF in any way.
Then there is the entire "alternate history" sub-genre of SF, like Flint's 1632 series or Turtledove's World War (how does Earth cope with an invasion of alien lizards in the course of WWII?)
And famously to the annoyance of fen, there are authors, like Margaret Atwood, who refuse to admit they write SF.
OT, but the first time I read Jane Eyre, I read it as SF, because at 10 y.o. I could not imagine a world without electricity. Worked for me, but I still preferred "Have Spacesuit, Will Travel".
I like the term "speculative fiction" which says that if the story postulates a currently impossible situation (terraforming in a new star system, lizard invasion, fundementalist religion combined with endemic infertility, whatever) and works from there, then we have speculative fiction.
And of course, there is always the problem that all SF is so rooted in the time it was written, that it dates more quickly than "mainstream" fiction.