Story, atmosphere and gameplay are the main 3 for me. I loved Corpse Party since I thought it incorporated all three of those aspects so damn well.
If you haven't played above game, grab it for PSP and play it at 2am with headphones on.
Graphics isn't as important to me. You could make the game text based and I would probably sit through it if the story and atmosphere were good enough.