Why forms ‘knowledge’ is important
Much organisational ‘knowledge’ originates with forms and forms data often comes from form-filler’s ‘knowledge’.
Bad design blocks ‘knowledge’ extraction. If a form’s data doesn’t INFORM, it isn’t ‘information’, just misinformation and misinformation means poor corporate ‘knowledge’.
Forms knowledge sharing—a two way process
The questions on the form are based on the organisation’s knowledge of what is required as well as the designer’s knowledge of how the form filler will respond. The form fillers then have to use their knowledge of the organisation, the organisation’s needs and their own information to be able to fill in the form correctly. Forms are a structured question and answer dialogue, generally without the opportunity for feedback and so are highly error prone.
The cost of errors on forms
The real cost of forms goes way beyond the cost of printing, storage, distribution and routine office processing.
It is commonly believed that since many people are functionally illiterate, designers can do nothing to produce forms that people can use without making mistakes. On the other hand many people think that form design is ‘kid stuff’—the sort of work you can give to a low-grade office worker—or something you can give to a graphic artist and as long as the forms look ‘pretty’ everything will be fine.
There may have been little excuse 20 years ago, but now we know better. Forms can be made to work, but to do so we have to clear our minds of preconceived notions of design, forgetting what we’ve been told for many years and coming into the 21st Century. It’s not that teachers of traditional methods have been careless in doing their job—they just didn’t know what we know today.
Patricia Wright was one of the early pioneers in forms usage research and she had this to say:
“Those who seek simple recipes for designing adequate forms have failed to understand the complexities of the problem.”
— Patricia Wright, Visible Language, 1980
As we move more and more into the area of electronic forms, these issues become even more important. A major cost is the time and cost burden on the processing staff that have to deal with the errors. Here are six examples of the cost of errors that I’ve had personal experience with.
Example 1
Each year 550,000 people filled out a government application form for financial assistance. Close to 100% of the forms had one or more errors resulting in most of them being sent back to the applicants for more information. Some of the forms were sent back to users up to 4 times to get correct information. We estimated that even the basic processing cost to correct the errors was over 2 million dollars per year; that’s only $4 per application. The real cost was estimated to be closer to 10 million dollars, and that was only ONE form.
Example 2
A State Government utility surveyed 2000 completed forms, half of which were internal, and they found 80% to have one or more errors. Again, the error correction cost was huge.
Example 3
A State Government Department had an application form for business incorporation that resulted in letters to 90% of the applicants asking for more information. They didn’t know the cost.
Example 4
I found an insurance corporation allocating 20 minutes per form in its work planning JUST to deal with errors—and that’s the errors they knew about. Even worse were the actions of the data entry operators. Because the documents were batch processed, if there was an error that would cause a batch to be rejected, the operators would key data that they knew would be accepted. These errors were never reported to management or the applicants.
Example 5
A government grant application form that had 97% of applications wrongly completed. After extensive analysis of the errors, the redesigned form was a vast improvement.
Example 6
This relates to two major life insurance corporations that each had close to 100% of their forms with one or more errors. As part of the redesign process we conducted extensive useability studies. Subsequently, it was reported to us from one of those companies that the error rate had been reduced to around 5% of forms with one or more errors. What was just as encouraging was that during testing, one complex 24 page form were being completed by applicants in under 30 minutes and virtually error-free. The solution was the design and testing methods that I’ll describe later.
Problems people have with forms
This section summarises a few of the more common issues. More information can be found on my company’s web site in the Free Literature section.
A major decision for form designers concerns whether to cram the questions on the page or to give many pages. First impressions of a form count, but so does the person’s experience as they progress. They may be initially bothered by many pages but this will usually change with a good form as they progress.
Users also need to see structure on the page. Lack of structure causes anger and confusion.
Most people approach forms based on their previous form-filling experience, which is usually bad. With a long form, as a person progresses they get into the routine of how to answer the questions. If you change the style part way through, they may make mistakes. For example, in a series of No/Yes type answers, it is normal for the ‘No’ to come first. If the user is half way through the form and the sequence changes with the ‘Yes’ first, many form fillers don’t read the caption and just assume that the ‘No’ is first. The same happens in reverse if the ‘Yes’ is usually first.
Most people don’t like filling in forms
Most people find forms confusing and intimidating and there is often frustration from lack of space. Not so long ago we conducted two major projects with usability testing on forms for aged people. We repeatedly came across aged people who had never—in a whole lifetime of experience—found a form easy to use. We found that they just expected forms to be bad.
Form fillers often find that the language and sequence doesn’t reflect their needs so they can’t see the relevance of the questions.
Many complain that forms take up to much time to fill in. Much of this is due to bad design and inappropriate language.
Then there’s the problem of lack of trust. Form fillers don’t trust the organisation. This has been a common problem with government forms. But it applies as well to commercial forms such as those provided by financial institutions and insurance companies. Form fillers often believe that the forms are trying to trap them. This is particularly relevant to insurance forms.
Fads
Fads have been a major part of form design, especially fads from the printing industry. Electronic forms are the current fad. Many organisations just place Word or PDF forms on a web site because it’s the ‘thing to do’ to save paper, with little thought to whether or not they really help the form filler.
A better solution would be to create proper intelligent electronic forms that help the user.
The problem of distorted views
There are 3 problems with distorted views.
- Form fillers: they often have a distorted view of their ability to fill out a particular form. Many people think they know how to fill in particular forms when they don’t. A typical example is tax returns which most people get wrong if they try to fill them out without professional help.
- Administrators: our extensive experience with public use forms has shown that many administrators don’t understand the public they are dealing with and how those people understand the forms. They think they understand, but evidence shows that they often don’t. The solution is for the administrators to take part in the testing process that is discussed later.
- Graphic designers: Many graphic designers are untrained in information design, not understanding the human communication issues. They know how to make forms LOOK good. They know how to use all sorts of creative ideas to impress the management of the owner organisations, but time and again we find that the forms just don’t work. That was a major problem with social security type forms we worked on for Centrelink, the Australian Government agency that processes these forms. There is much more information on our company web site about this project. The forms looked impressive and included some interesting creative graphic design ideas, but in the final analysis, the number of errors was costing the Australian Government millions of dollars per year to correct.
Good forms are possible
It is possible to guarantee successful forms. This doesn’t mean that all forms will be 100% accurate, but error rates should come down to as low as 5% with one or more errors.
To achieve success, at least two things are necessary.
- Best practice: forms should be designed according to ‘best practice’. Go back thirty years and our knowledge of what made a good form was severely limited. But today there has been a large amount of research and we know how to design good public-use forms.
You will find a lot of information on this in my book Forms For People and in Caroline Jarrett’s book Forms That Work.
- Usability testing: the next step is usability testing, which I’ve covered in more detail below as well as in Forms For People.
In most cases we also recommend error analysis of existing forms to determine where the problems are.
Usability testing
Traditional methods of ‘testing’ include opinion surveys, pilot studies, readability scores and focus groups. But for the most part, they don’t TEST forms, they only provide opinions or inaccurate recollections. They often concentrate on treating people as machines and ignore the mind.
One of the most useless techniques is readability scores such as the Flesch Reading Ease Scale method. We have an excellent paper on this downloadable from our company’s web site.
Another useless method is focus groups. Many people place a lot of ‘faith’ in focus groups, but they provide little useful information for forms usability. Again, I have a lot more to say about this in Forms For People.
Modern research methods show the form in action and show us WHY people make mistakes. Most of the methods mentioned above don’t TEST forms to find out whether or not they are actually working. They concentrate on treating people as machines but ignore their minds and the complexities of their social interactions.
To produce quality forms we need a different approach—one that lets us see the forms in action and work out in advance if the form is going to work. We need a method that give us empirical evidence about their form filling behaviour—why users make mistakes, why they don’t carry out what was expected of them and the problems they face.
For our purposes, behaviour includes:
- The way in which the person carries out the task
- Physical things such as turning pages or moving through the document
- Facial expression and other mannerisms that might indicate problems, frustration, lack of understanding and confusion
- What the person says
- Most important of all: finding out as much as possible about how the person understands the document. What is the cause of any misunderstanding? Do they give answers to form questions that the processors correctly understand? Do they carry out instructions or do what is expected with the information given?
Observational studies are a method whereby you can find out why people are going wrong—where you can highlight specific user problems and fine tune the design to get rid of them. Using structured observational studies we watch users filling in or using the forms and, with appropriate questions, we can learn why they make mistakes. We learn about their real requirements, what they really need and want, and we collect information about their behaviour when using the form. The aim is to study the document in action in an environment as close as possible to the real world. We don’t just want to know what people think of the form or how they think we should ask the questions. We want to know about their behaviour—what really happens when they fill out the form.
One of the most valuable aspects of observational studies is that you can actually SEE the form improving through the testing stages. They also provide a great amount of fine detail and yet they are relatively inexpensive.
While each round of testing uses only a few people—perhaps 6 to 10—over the course of the study these can add up to a large group.
Conclusion
It is possible to have good forms and to collect accurate information from form fillers. This in turn leads to much more accurate knowledge for the organisation—a factor which I believe is an important ingredient of knowledge management.
——————————————————————————–
