Many important traits for a successful play are not exclusive to theatre, but to art in general. Presenting some form of truth, giving you something to talk about (great insight by Bowman in his post), taking risks with accepted limits, making an opinionated stand, inspiring people to change their lives, conflict -- all of these make for great movies too, and great songs, novels, poems, paintings, photography and whatever else you can think of (not so sure about pottery, though). In order to focus on what makes good theater, and not good art in general, I'll start with Jenny's definition of play as the work of art that doesn't allow you to rewind, go to the bathroom etc, just like Andy did.
I think the logical conclusion to that rationale is that successful theater establishes a kind of connection with the public that is not possible when you can rewind, pause or look away -- a connection based on the one moment in time shared by everyone in the space of the play. It is engaging, but with an intensity different (not "higher": different in nature) from a recorded performance or a written page. Perhaps because of the ephemerality of a theater performance, that can never be repeated and will only survive in the viewer's memory (and the reviews the next day). I'm thinking of that cliché that says the fire that burns brightest burns fastest, though, in the case of theater, it seems to me that the speed of the burning causes the increased brightness (or is at least a major factor in it).
Some of our readings provided very punctual insights on this connection between public and performance. Greg Allen's Rule #12 ("Do not suspend your audience's disbelief") is really deep stuff. On the surface, it seems to contradict the established notion that the acting must be believable, but perhaps plays are more engaging when you do realize what's going on, and that you're a part of it (as opposed to just sitting back and following a story, forgetting that you are watching a play). We get further glimpses of this metalinguistic awareness of the production when Edward Sobel says "embrace the unknown" (last paragraph) and Allen's Rule #15 calls for surprising even the performers. For instance, when a stand-up comic reacts to the audience (e.g., to a heckling comment or a particularly conspicuous laughter), he's abandoning suspension of disbelief in the interest of a more immediate rapport with the public. It's really not that different from a Shakespearean character sharing his thoughts to the audience in an aside comment.
I also want to bring up the sexual bit. It sounds like a joke, but I think there's something big there. It's been brought up too many times to be ignored: Lilywolff's piano, Sobel feeling like having sex with everyone in the room, Rivera's emotional extremes (assumption 24, and maybe 15 as well) -- it might be the main reason for feeling a part of what is going on, for a production being "so moving that I don't want to leave the theater (Bruce Seawerth, p. 6 of the course packet). That "visceral" connection might be more literal than is usually intended. I've felt that a couple of times when watching plays (and in some of them there weren't even naked people onstage). It changes your heartbeat, your breath, makes you aroused (I don't want to kill Lilywolff's boner by attributing it to dilated blood vessels, but it's a possible cause). It's difficult not to think of the origins of theater in Ancient Greece, with inebriated followers of Dionysus breaking into noisy orgies to win the God's favor and a bounty of fertility and ecstasy.
At the same time, I don't get a boner out of every play I watch, and some of them are very engaging as well. In my experience, the most captivating plays are the ones that suspend not necessarily disbelief, but time: the action happens in an endless moment, and when it's over it's hard to tell how much time has passed (and often surprising when you look at your wristwatch and find it out). It's absorbing. I'm positive that feeling like not leaving the theater is a constant in successful performances. And that doesn't necessarily demand some sort of sexual response.
But there's definitely something big there.