A Novel Approach?

Like everyone else on this blog, I am trying to write a novel. But for my “day job”, I write computer software. I work for a grocery chain and we write the systems that help them run the company. At first glance, writing novels and writing software don’t seem to have much in common other than the verb “writing”. Writing novels is creative and writing software is disciplined, right? Well, yes and no. Writing software is actually very creative. And writing novels is turning out to take a lot more discipline than I realized.

The more I got to thinking about it, the more I realized that there are a lot of parallels between writing novels and writing software – and that some of the lessons I’ve learned on the job could actually help me in my novel writing.

Let me explain.

Computer systems are big, complex things and we software developers have always struggled with how to deal with that complexity. When you first approach writing a new computer application, first there's the problem of understanding just what it is that the system users want it to do for them. Then there's the problem of translating that to technology, then actually constructing it, then combing out all the errors. If you make it through all that, you have working software that's of use to someone. If you don't, you have a failed project. Studies have indicated that as many as 70% of software projects fail, collapsing under their own weight as they grow. I don't think it's that high, but there are definitely a lot of failures.

Novels are also big, complex things, and the stages of writing a novel actually seem to parallel the stages of writing software. The "requirements" stage of software development is a lot like mapping out the plot and developing the characters of a novel, trying to put together something that the users (readers) will want to read. The "design" stage is like mapping out the scenes in detail, and the "coding" phase is the actual writing. The "testing" phase, where defects are found and corrected, is the editing and revising phase of the novel.

As I work with my writing group (and remember other groups I’ve had in the past), it’s clear that, like software developers, we’re all struggling with that complexity. And I’m curious what the failure rate is for novels that get started and never get finished. I’ll bet the success rate is in single digits.

Back to software. In order to alleviate the high failure rate, software developers have invented and tried a variety of methods over the year, all aimed at managing the complexity of the projects. For most of my career the "waterfall" method has held sway. The idea behind this method is that each stage of the development lifecycle must be completed before the next one is undertaken. For instance, all the user requirements must be clearly understood before any technology design work is undertaken, and all the design work must be finished before the actual coding starts. This method was based on the principle that an error is much easier to fix on paper, in the requirements or design stages, then in code or, worse yet, during the testing phase of the project.

The biggest problem with this method was that it was almost impossible to get the requirements anywhere close to correct, meaning that the remaining stages were built on faulty information. The reason it has been so hard to get it right is pretty straightforward: the future system users typically don't know what they really want. They have a vague idea, and they may articulate it as best they can, but they don't really know if it's right until it's actually built and they can put their hands on it and try it. Then they don't have any problem identifying what's working for them and what's not.

If the team followed the waterfall method, it could be many months or even years between the time they first start talking to the users and the point, after all the requirements and design work is done, when they actually start coding and giving the users something they could see and work with and give real feedback on. And if it wasn't right, they would have to rip pieces out and start over. Plus, the whole process took so long that the business might have actually changed while the software was being written, and it would be obsolete before it was delivered.

Because nobody wanted to start coding until they’d gotten everything absolutely right in the design, the project could easily bog down in the design process. In the search for perfection, the analyst would never feel they were really done with their phase. There is even a name for this condition: “"analysis paralysis.”

The novel writing community has coined the terms “plotter”, for writers who plot everything out meticulously before they start actually writing, and “pantser”, for those who jump in and write, navigating by the seat of their pants. In my analogy, plotters are using the “waterfall” method of novel creation. And I do believe that it’s easy to get stuck in analysis paralysis in that mode. And, for me, anyway, after putting in all the effort of outlining my plot, I finally start writing, only to find the story veering away from my plan as early as the first chapter. What looks good in an outline doesn’t always work in an actual book.

This may drive some plotters to make the leap to being pantsers.

There is a software equivalent to the “pantser”. (We call them cowboys.) Software developers who saw the waterfall method fail would frequently react by throwing out the whole methodology. They would instead have only the briefest discussion with the users and then start coding. They would show the users what they'd done, get the feedback, and rework the code. This wasn't a particularly satisfactory method either, though. For one thing, typically it meant a great deal of rework since the developer's guesses were rarely on the mark, or because a requirement discovered later in the process would completely change the underlying design, so it would have to be ripped out and done again. Perhaps most importantly, though, it's almost impossible to put together anything really complex with that kind of haphazard approach. You may be able to build a simple log cabin without a blueprint, but you can't build a skyscraper by simply stacking up a bunch of log cabins. And a computer system is more like a skyscraper than a log cabin.

A novel is more like a skyscraper, too. So where there may be a few brilliant “pansters” who can just sit down and let words flow and end up with something cohesive and readable. I would wager that most of us can’t.

What’s writer – of software or novels -- to do?

In the last few years, a new methodology has surfaced in software circles, one that tries to walk the line between planning everything ahead of time and doing no planning at all. It's called “agile software development”, and it goes like this:

1. Understand the system requirements end-to-end, but only at a high level. Map it out, but not in detail. It will change as the project progresses.
2. Establish a design framework. This may include decisions about what programming language the software will be written in, and its general organization.
3. Pick a slice of this to work on first. It doesn’t have to be the “first” piece – for instance, the first screen the users see after they log on. It could be the most important piece, or the most technically difficult piece.
4. Design this piece in more detail, keeping the bigger context and the design framework in mind at all times.
5. Code this piece. Don’t worry about the bugs (software defects) at this time. It doesn’t have to work perfectly. It just has to work well enough to show it to the users for feedback.
6. Show it to the users, then take their feedback and work it back through the process. This may mean adjusting the requirements a bit, adjusting the design a bit, and doing some re-coding.
7. When the users like what they see, pick another piece to work on and go back to #3. Note that with each piece of the project, the developers gain more information, some of which may be worked back into previous pieces or even into the overall framework.
8. Once you have the functionality of the system going the way the users want it, then go back and work out the bugs.
9. When the software is as defect-free as you can make it, deploy it to the users.

I’ve been using this on the job for a while and it’s working like a charm. It’s not that we don’t get things wrong any more; we do. It’s just that everyone expects it to be an incremental process with missteps along the way. The first time we show software to the users, they don’t expect it to be perfect. They just expect it to be something they can give us feedback on. They know it will keep getting better as we go along.

I’m reminded of a quote from Thomas Edison, who apparently tried and rejected something like 10,000 designs before he perfected the light bulb. When asked how he felt about 10,000 failures, he said that they weren’t failures – that inventing the light bulb was simply a process with 10,000 steps.

So with this in mind, here’s my suggested guidelines for “Agile Novel Development” (maybe I’ll trademark it!):

1. Understand the plot, theme, and characters of your book end-to-end, but only at a high level. Map it out, but not in detail. Know from the beginning that it will change as you go along.
2. Establish a design framework. Who are the POV characters going to be? What is the target length of the book? Will it be dramatic or humorous? And so on. These are guidelines that will help you as you go along – and once again, they will change.
3. Pick a slice of the plot to work on first. It doesn’t have to be the beginning of the book. If you can see the “black moment” or most pivotal scene in detail, for instance, but aren’t quite sure what led to it, then write the black moment. Or write the piece that excites you most, or the one you think you will have the most trouble with.
4. Work out the scenes for this piece in more detail, keeping the bigger context and the design framework in mind at all times.
5. Write this piece. Don’t worry about defects at this time. It doesn’t have to be perfect. It just has to be good enough for you to see whether it’s going to work. Depending on your style, you may also be planning to share it with a critique group, but only for high-level feedback. (This means the critique group has to be very aware of where you are in the process, and keep their feedback focused on what is relevant for that stage.)
6. Review it and / or show it to your critique group, then take the feedback and work it back through the process. You may have discovered something new in writing this piece that alters the plot, gives you more insight about a character, or even changes the theme or genre of the book. This may mean adjusting the plot, characters, etc.
7. When you are happy with that piece, pick another piece to work on and go back to #3. Note that with each piece of the project, you’ll gain more insight into the story, some of which may need to be worked back into previous pieces, or that may even change the overall framework. This is expected to happen! It’s not failure, it’s part of the process! Don’t get frustrated – expect it and even enjoy it, if you can.
8. Once you have the first draft written, go back and work out the bugs. Remove wordiness, add foreshadowings, proofread, smooth out the prose, punch up the dialog.
9. When the software is as defect-free as you can make it, deploy it to the agents and editors.

If anyone has made it this far, congratulations and thanks for reading! Hope there was something in here that you could use.

Happy writing!

Samantha