I must admit that I no longer remember much about the story but the essence is that a human crew (including women and children) has been taken hostage (aboard a space ship, I believe). The aliens operate without an unconscious and so humans who do have one are easily handled by the aliens because of the lag time when the unconscious thought moves to the conscious and so the Aliens always react faster to thwart the humans.
I believe you are seeking the short story "Whatever Counts" by Frederick Pohl from 1959 Galaxy Magazine. Reprinted in the "Abominable Earthman" collection in 1963.
The Abominable Earthman, (1963) "The Abominable Earthman", 1961
"We Never Mention Aunt Nora" [as by Paul Flehr], 1958
"A Life and a Half", 1959
"The Martian Star-Gazers", 1962
"Whatever Counts", 1959
"Three Portraits and a Prayer", 1962
In it, a human colony vessel attempting to establish a colony on what is believed to be an empty planet encounters an alien species that is hostile. The aliens appear to move unbelievably fast by human standards. The ship's doctor/psychologist discovers that the aliens have no subconscious, hence do not require translation of though through any lower brainstem levels. There is only one time that the doctor observes that the humans are able to outmove/out-maneuver the aliens. That is when the aliens have threatened the children on board the ship and the humans reacted by harming/killing the aliens.
To prevent the aliens from punishing the humans and keeping them alive, the doctor then tries to prove to the aliens that humans have a subconscious. In one experiment, the doctor shows a woman an increasingly difficult set of some type of test, each time rewarding or "punishing" her with an electric shock. At one point she is unable to consciously determine what the correct answer is. However, the doctor has her try anyhow under hpynosis. To her amazement it works, showing that somehow her subconscious is providing the correct answer, though consciously she cannot perceive the correct answer.
At the story's end we find hat during the course of showing the aliens that humans have a subconscious, the doctor has been concurrently using hypnosis to train the crew on how to escape. In essence, to "act without thinking" and doing it so fast, they can outmaneuver the aliens to escape.