1. Not Enough Happens!
You'd be amazed how often this throws up an obstacle to publication. Literary magazines are not pulps that require a murder every three pages, but a lot of stories that come in are weak because they simply don't have any action at all. Hundreds (and I do mean hundreds - I read 60 stories a day, three days a week, for three months) of the stories I read had competent writing, but absolutely no action at all. Instead, the reader followed a character's pottering around his kitchen thinking about how he hates his job for five pages, or a guy went to work and performed his job while daydreaming, then went home. And that was it.
After five or more pages of this kind of story, I was ready to put it in the reject pile without even finishing it. It's important to remember that stories are about change and development, not just about giving a snapshot of an ordinary day in an ordinary person's life.
After the jump: more reasons why your story is getting rejected.
Literary magazines have very limited space. They don't have room to publish everything that they get that is well-written. So one of the questions I was taught to ask myself while reading was, "What is unique about this story?" In other words, what is this story doing that I've never seen before?
Too many times, I've run across a story with adequate writing and a fairly well-drawn character -- but the story and character is cliched, utterly un-unique. Whether it's the boozing abusive husband or the suffering housewife, the characters seem two-dimensional because they are types, with nothing individual about them. The plot -- quietly struggling marriage, tempestuous affair, murder mystery -- has been done before and better. This is the second fastest way to get your story a form rejection letter.
3. The Writing is Awkward or Weak
No matter how fascinating a subject you've decided to cover in your story, it won't make it past the first page or two if the writing is anything but air-tight. Literary magazines exist not to give people thrills and plot twists, but to showcase the best possible writing they can find. That means writing that is vivid, economical, smooth, and readable. To improve on all of these fronts, try reading your work aloud to yourself. It becomes dramatically evident which parts are boring and wordy, which parts are confusing, and which parts are just clumsily worded. Polish those areas if you want a chance of getting past the first page with a reader.