Seems like this is the most powerful weapon of all: transport your enemy into a bulkhead, planet, etc. Transport your enemy's ship inside a planet, etc.
Is there some technical reason that it can not be used this way?
Your basic assumption is correct. The transporter is "potentially" one of the most dangerous technologies the Federation and its allies could ever have. You could:
I can think of a few others with only the tiniest bit of effort. So what stops this from happening? The primary defense against transporter technology is the shield technology used by nearly every race in the Galaxy. The few races that do not use shields have a hull that prevents locking on by using dense hulls and powerful electromagnetic energy fields within their hulls, i.e Species 8472.
The transporter requires a targeting lock, the ability to ensure transport between two locations, so the shields prevent such a lock and the ability to prevent the lock does not require the full power of the shields. As long as any power remains able to be diverted to shields, no transporter actions are possible. This is one of the most basic tactics in Alpha quadrant combat. Drop the shields, transport marines to the enemy ships. Klingons prefer this and the Dominions Jem'Hadar were also quite good at it.
A second reason is the difficulty in using the Transporter with the accuracy required for warfare. It has always been a technology that did not lend itself to crisis. It requires significant computing power, consider skill to lock and transport materials, particularly against moving targets (go New Chekov!) and considerable engine power to ensure effective transport. All of these things are less available when the ship is under fire. The transporter is also vulnerable to a host of environmental conditions, electromagnetic disturbances, ion storms, nucleonic fields, dense metals and rock and is limited by range 40,000 kilometers and relativistic effects (can't transport from/to warping vessels).
More important to the Federation is its ethics in combat. Though I cannot think of what the equivalent of the Geneva Convention might be (Seldonis IV Convention treaty) I am certain the Federation has rules about how combat should be fought and what weapons and techniques should be used. We know that biogenic weapons were outlawed and only used by rogue agencies (The Maquis, for example, who created a biogenic weapon for use against the Cardassians).
The transporter is probably one of those tools that does not get used the way it could because many of the techniques would be considered "unsportsmanlike" since it was a common policy even among more aggressive foes to take and exchange prisoners. So, combat was usually a thing where once ships were rendered incapable of fighting, surrenders were declared, life support and medical attention was supplied and prisoners were taken. Anything other than this ends up being a war crime in the very structured society that is the Federation.
I imagine other races may not play as fair as the Federation or in periods of "no holds barred" survival against such enemies as the Borg or Species 8472, I suspect even the Federation may take a "whatever works - by any means necessary" approach, and even that may vary from Captain to Captain.
When Captain Jean Luc Picard had the chance to create a recursive algorithm against the Borg, using the drone called 3 of 5 (renamed Hugh) that could potentially destroy any collective group that came in contact with Hugh, he refused despite the wartime footing of the Federation against the Borg. He considered it improper to use a living person as a weapon of mass destruction. (TNG: Descent, part I and II)
It is. Voyager used the transporters once to beam a photon torpedo right on board a Borg vessel. It was ... efficient.
Shields block transporters.
In general, transporters could not be used while the deflector shield of a ship was active, or a deflector shield was in place over the destination.
Taken from Memory Alpha.
This limitation would make transporters ineffective against any opponent worth fighting.
Finally, what would you use transporters to do? Beam someone into space? Scatter their atoms across a continent? Tremendously effective, but also inhumane. When possible, a phaser set on stun would be considered preferable by the peace-loving crew of the Enterprise.
A few people have mentioned shields as a big problem, that the enemy ship would have shields prevents it or nicely stated in Thaddeus Howze's answer:
The primary defense against transporter technology is the shield technology used by nearly every race in the Galaxy. The few races that do not use shields have a hull that prevents locking on by using dense hulls and powerful electromagnetic energy fields within their hulls, i.e Species 8472.
Another issue with this is that you would have to drop your own shields to do the transport, leaving you vulnerable to attack.
In the Next Generation episode "Captain's Holiday", Picard instructs the Enterprise to use "Transporter code 14" - the use of the transporter as a weapon by dematerializing an object and simply never rematerializing it.
Not only can the transporter be used as a weapon, there is an in-universe name for it.
O'Brien shows how a micro-transporter could be attached to the muzzle of a TR-116 rifle to beam the bullet close to the target, where it continues its trajectory. With an exographic targeting sensor, the murderer could have scanned through bulkheads, meaning he or she could be firing from anywhere on the station, at anyone. Unfortunately, the micro-transporter does not leave enough of a transporter signature to track back.
One of the most creative uses of a transporter in Star Trek, and points out just how devastating a weapon they would be. This is one of those episodes where a weapon is introduced and never heard from again. I suppose Starfleet considers assassination and sniping "distasteful", but other races and organizations sure wouldn't.
There is no reason disposable transporters wouldn't orbit around a vehicle outside its shields.
Additionally, as long as the target is in range, there is no reason to worry about the spontaneous reassembly on the 'other side'.
I think these transporter weapons (or should I call them de-materializers)? would be more inline with federation ethics. Yes, shields are a problem, but once they are down, using your transporter to just translate half the enemy into subspace energy would be much cleaner than blowing them up and creating a dangerous shrapnel field that will obstruct un-shielded fledgling space-fairing civilizations.
If there is a subspace race that will get angry at us, we can just store their ship in our buffers a-la Scotty on the Dyson Sphere.
Considering we see the ST universe from the view point of the Federation (And occasionally Cardassians or some of the one-shot Voyager Races), it's all colored by the Federation's ethical standards. This isn't new BSG, the Federation wouldn't even dream of Airlocking someone the traditional way, let alone using teleporters. They have no problem kidnapping someone ship to ship (to the bridge or brig), but teleporting someone into Space, a Bulkhead, or Rockbed is particularly gruesome and cruel.
And it's not just the Federation that would have that sentiment. Klingons, the single-faceted warrior race they are, would consider it dishonorable, a cowards tactic. They will beat you to death with their bare hands, but wouldn't space you. Cardassians have some (twisted) sense of honor and adhere to their own rules, including not torturing people if a treaty prevents it (Picard refused to admit he was working under Fed orders). Spacing someone only serves sociopathic purposes, and while I doubt they wouldn't use it as a threat, actually going through with it holds no logical or useful purpose. Same with Romulans. Both are pragmatists, not sadists. At best, occasional use to break the spirits of others, or to dispose of evidence. The Borg arn't so imaginative.
Then there is the practicality. With Shielding and other disruptive technology limiting when they can use it and the Technical Effort needed to pull it off, you could just throw some incredibly powerful missiles or phaser beams at them. Why use a difficult to use nuclear-powered scalpel when a easy to use nuclear explosive works?
Finally, there is the production reasons. This is a family show.
Creatively speaking, Teleporters are a very versatile tool, let alone weapon. They could start the teleportation process and stop it mid way, allowing the person's patten to degrade and fade away. Or teleport poison/virus/carcinogenic directly into someone's stomach. Teleport their organs away. Or teleport a cloud of poison/sleep/acidic/plasma gas into a room. Long Range Lobotomies! Teleport invasive fauna (read: tribbles) or mold or wild animals or nanite infection. Teleport one person INTO another or blood right on them. Teleport an entire bee hive or spider nest into a person. Some real nightmare fuel scenarios. Think Saw in space.
A variety of reasons.
Can't penetrate force fields, starship shields... You think a planet doesn't have some sort of protection against it?
According to Wikipedia's article on Transporter (Star Trek): Capabilities and limitations.
Most of the planets where we have seen fighting in the ST universe have no shields.
As a weapon, in the case of being surrounded by say an alien enemy squad, and you wanted to make a strong impression on them, without necessarily killing them, may I suggest you simply transport them 10-20' into the air. Or a holding cell. Or if you feel a bit more draconian, try transporting body parts to places they don't belong. That'll make an impression on your enemy if it doesn't kill them.
OK. Let's forget its use as a weapon, and rather more in terms of the transporter as troop support in an extended battle. How 'bout if you were in the middle of a phaser fight and unable to find time to eat? Instant lunch in your tummy! The transporter with an appropriate scanner may also be a good way to relieve yourself. :) The comic possibilities alone are worth the writers' consideration.
I would say that small antimatter bombs with a moderate yield would be swell for ripping down an enemies shields. Then the enemy vessel could be rapidly bombarded with photon torpedoes phasers or more anti-matter ordinance. Ethical concerns be damned.
There are some instances of transporter weapons being used, one in an early episode of voyager where the lungs are transported out of a person, and another in TNG where (Picard or Riker) is transported out of a bar and made to look like he was completely vaporized instead. The other answers address why it's rare/uncommon for the federation or its allies to use such weapons.
in voyager the Voth city ship transported the whole ship shields and all. sure it was huge but if an old bird of prey can transport a whale then scaling up to a ship shouldn't be that hard. also remember the borg can transport without worrying about shields
Ok, everybody is saying here shields and so forth but what about cloaking device? We see all of the ships having to turn shields on before battle, implying that they can not be always on. So if you have a cloaked ship, you can beam torpedo or some sort of timed ordinance in to the ship that does not have it's shields on.
To me that is a major, major weapon. Probably most devastating one. Even if you get detected after beaming something all you need is one shot and then you can bolt out of there.
Imagine hit and run tactics against anyone this way. Imagine what 10 cloaked ships could do. Destroy ANY 10 targets and bolt.
Rinse and repeat.
You are all forgetting the real reason.
Anything else is just excuse making. It makes no damn sense not to use a transporter as a weapon and as has been pointed out here, they are no consistent rules that would prevent such.
The writers wanted a plot device that would skip expensive model shots for landing on planets and found a cheap effect that looked really nice. It doesn't have to be scientific or logical, it's just a plot device.