Despite what you may have read as a child, books can have unhappy or not-so-happy endings. An ending that isn’t so happy is one where at the end of the story, your main character never gets anything that they were aspiring for.
Stories with unhappy endings are much like math problems in which the final answer is a negative. Endings may have a silver lining, or something redemptive, but generally, in an unhappy or not-so-happy ending your protagonist comes out with a loss. The loss could be so severe that it’s a huge victory for the antagonist or it could be a revelation of the inner character of the protagonist.
What about a tragic ending?
If we think about the quintessential unhappy endings, Shakespearean tragedies, we could conclude that a tragedy is an unhappy ending, just stabbier. Actually, a tragic ending is more complex than that. Tragedies have a dark tone, from beginning to end. The journey of struggle and loss, despair and defeat, needs to come out in every scene. Even if, like Shakespeare, you have moments of comic relief, you need to keep things dark.
What about a twist ending?
Endings with a twist is where something unexpected, but possible, comes to shake things up. How to get this right? From the very beginning of the book you should know what secrets to keep from the reader. This could mean the identity of the real killer, or that the narrator is dead, or that the sidekick was working against the hero. How to get this wrong? You slap the ending on at the end, just to get it over with. It’s unexpected, but it’s not really possible. This isn’t a twist, it’s just lazy writing, and it’s a sure fire way to get your readers to hate you.
What about a loop ending?
In a loop ending, your main character comes back to the place where they started, either emotionally, geographically, or figuratively. How to get this right? It may require you to rewrite your beginning several times so you can see two bookends that are congruent to each other. I’d argue that a loop ending is probably the easiest to pull off because you’ve already been there once — you can have a loop that has a positive spin or one with a negative spin.
What about an ambiguous ending?
In endings like this, nothing is really resolved and the story just kind of stops, usually with some distinct image, like the green light in The Great Gatsby. The main character may or may not achieve the goal — this isn’t clear and it’s not supposed to be. Most of the time writers who choose this are more interested in the last image and what it represents, or perhaps another symbol, or the emotion that is left with the reader. I would argue that classic examples of this were carefully written, not just randomly stopped. I think that writers who choose this way do it out of style, symbolism, or aesthetic choice, not to tie the plot up in a tidy bow. I would also argue that your genre and tone would have a lot to do with your choice here. The more light-hearted and genre-specific your book, the less likely an ambiguous ending is going to work.
What about a cliff-hanger ending?
This is when something is left unresolved for the main character. How to get it right? Have a sequel in mind. Writers of sequel books use these endings so that readers pick up the next one in the series. It works! How to get it wrong? Maybe by not making the reader care what happens. — The unresolved situation isn’t that big of a deal or it’s poorly executed, or it feels like the author was lazy.
Tips for writing every kind of ending:
Stay on track. The further you get away from the first act, the less you need backstory, new characters, or new tangents. Your story needs to be tight, for as you progress, you don’t want too many loose ends to tie up.
Keep the protagonist in control. Your main character must make choices that bring them closer and closer to a resolution: whether or not they will meet the goals set up in the beginning of the second act. Ideally, each victory and defeat in the progression of the book increases the tension; the stakes are higher and more can be lost if they fail. You can’t have your main character give up, pass the action onto another character, or surrender to the antagonistic forces.
Don’t let your protagonist be a disappointment to your readers. They’ve been cheering your main character on for two acts, so reward them by keeping them in control, even if they don’t accomplish their goals.
Avoid the obvious. Your readers will enjoy a surprise or two along the way. Before you get to the ending, make a list of at least twenty possibilities then choose the most unusual one. Then, work backward. Make notes on how you can set this situation up. Make a list of necessary points, like information being revealed, certain antagonistic forces foiled, certain weaknesses exposed.
Think cause and effect. Your main character must be both proactive and reactive to the situations around them. Ultimately, they have to come to the big climactic scene where everything is at stake. Regardless of the ending you have in mind, the conclusion of the story is the result of all the preceding choices.
Think about a permanent change. Stories need a sense of finality to them: an ending that’s truly a turn or a conclusion. Ideally, your characters have changed throughout the narrative and by the end it will be obvious to the reader that they can’t go back to what they were.
Like happy endings, unhappy (or not-so-happy) endings need to be believable and plausible. With these tips, your characters can shuffle off into the sunset, or not, and your readers can be satisfied.