Teaching by Example
In early 2002, a White House promise spawned a program that would radically change the way civilians interact with the federal government. In speech after speech, White House officials proposed creating an Office of Citizen Services to help people find government information and services. As that goal shifted from rhetoric to reality, it became clear that the Web and e-government would play a primary role in its fulfillment.
The responsibility for launching the new organization, first tossed to the Office of Management and Budget, eventually landed in the General Services Administration’s Communications Office. Choosing GSA made sense because in 2000 it created the government’s main Web face to the public, FirstGov.gov, and also had purview over the central citizen help line, 1-800-FED-INFO.
Federal Guinea Pig
Now, tasked by OMB with the pumped-up mission of setting new online best practices for agencies governmentwide, GSA’s Office of Citizen Services and Communications (OCSC) is using its efforts on FirstGov and the telephone help line as a test bed for usability standards that will apply to all agencies, and be recommended to state and local governments as well.
“Many government sites — even those with a much more limited mission than FirstGov — have to support users who have very little experience with how an agency or department is organized,” points out M.J. Pizzella, associate administrator of OCSC. It’s that reality, she says, coupled with her office’s experience organizing thousands of links to government services and information, that places her GSA team in the unique position to help others in government create user-friendly sites.
Over the years, FirstGov has been inching up the satisfaction ratings tallied by ForeSee Results (see story here). The Ann Arbor, Mich., surveyor uses the American Customer Satisfaction Index (ACSI) to rate the public’s likes and dislikes when it comes to electronic interactions with government. GSA has seen FirstGov’s rating, which currently stands at 72 out of 100, improve by a decent five points in just two years. Pizzella hopes that a major revamp of the site, scheduled for late spring, will push the ACSI rating still higher.
Relative to other agencies’ sites, FirstGov is middle of the pack when it comes to pleasing users; the aggregate score for all ranked federal sites is 73.5. Two Social Security Administration sites share the top score of 86 in rankings that ForeSee Results issued in March: One hosts SSA’s online benefits application and the other offers guidance on the Medicare Prescription Drug Plan. Compared to the big industry-run portals — AOL, MSN, Yahoo and the like — FirstGov holds its own. These portals received an aggregate ACSI ranking of 75 on the latest ForeSee commercial e-commerce ranking.
This spring, OCSC will submit a new set of proposed Web site usability practices to OMB. There is a current set of recommendations created by the Interagency Committee on Government Information for OMB. But it’s two years old.
Pizzella says that agencies should expect that some practices will be mandatory while others will be recommendations or, as she likes to say, “need to do versus should do.”
“We are kind of the usability gurus,” she says. The administration has asked the FirstGov team to help agencies figure out how to provide more online services and to keep their sites clean and well run. “We are being given more and more responsibility on this by OMB.”
User Think
If there is one theme that runs through GSA’s Web design philosophy, it’s the importance of understanding the users. “Many sites, including our early ones, were built around the way government people think, not the way citizens would expect to find information,” Pizzella says.
business for the citizens,” GSA’s M.J. Pizzella says.
To make her point, Pizzella recalls the rude awakening the FirstGov team got during a focus group in 2002. GSA had brought in 15 diverse people, some government employees and some not, to test the site. When asked to navigate the site and find links to the FBI, only one tester succeeded — and she worked for the Justice Department. The problem? The FBI links fed into the site under a hierarchical menu whose top category was Justice, a connection unknown to many citizens. To solve such problems, OCSC added a horizontal A-to-Z browse method to supplement the hierarchical one. Citizens can go directly to the FBI link via the alphabetical listing.
“One of our jobs is to remind the agencies that we’re all in business for the citizens,” Pizzella says. People used to cut the government some slack on its online information and services, she says, but no more. “They want more information quicker, faster.”
While an A-to-Z index is great for people who know the agency they need to contact, it’s of little use to people who want a service or information but don’t know the government entity that offers it. To meet the needs of those people, OCSC provides functions that accommodate two types of information seekers: browsers and searchers.
For browsers, a hierarchy would help people narrow their browsing behavior until they reach the link they need. But the hierarchy should not depend on an understanding of government organizational charts. Pizzella says the best way to organize content for such visitors is by type of user. So OCSC classifies users into four categories: citizens, businesses and nonprofits, federal employees, and government to government.
The site also has a few more specific audience segment labels, including kids, parents, seniors, and military and veterans.
In addition, FirstGov visitors can browse by topic or news.
Beverly Godwin, director of the site’s content, says to make data readily accessible, the same document often requires links under multiple headings. Although that process entails some extra work in determining all the ways people browse for information, Godwin believes it is time well spent. Agencies need to test how visitors to their sites look for information and then “make sure the information is placed where they expect it to be,” she says.
Power Play
OCSC has also beefed up FirstGov’s search facility with the help of technology from Vivisimo of Pittsburgh. First, rather than depend on Google-like searches, OCSC has put intelligence in search results by incorporating information it expects people will be looking for. For example, in the old method, when people searched for “Social Security,” they’d get a link to the Social Security Administration and some related Web sites. Now they receive links to frequently asked questions related to Social Security that the federal hotline staff compiled, links to Web sites recommended by the editors at MyMoney.gov and links to Social Security forms.
In addition, the new engine clusters results and categorizes them by topic. If a user types in “disability” and opts to view folders by topic, the search would return a list of the following folder names along with the number of items in each: Insurance (19), Social Security Determination (17), Aging and Disability (11), Disability Rights (11), Health (10) and Rehabilitation (8).
Besides categories, users can also view the results by source or agency.
One reason OCSC spends so much time seeking better ways for users to find information is that FirstGov goes beyond providing links to federal government resources and includes links to services and information offered by state and local entities, too. Although this adds greatly to the number of links and complicates the creation of appropriate search processes, Pizzella says other agencies should not shy away from expanding their sites by linking to resources for which they are not responsible. “It’s important to determine if citizens might expect to find certain information on your site,” Pizzella says. If they do, but the information or service is actually at the site of another federal, state or local agency, include a link to it, she says.
Pizzella warns that agencies need to regularly weed out dead links, which will occur if a federal site offers links off its main host. If a site has too many nonworking links, even if they are to other sites, users will consider it out-of-date and untrustworthy.
In the Now
Currency of site information is a fundamental that agencies must focus on, Pizzella says. And being able to post information quickly plays into it. Organizations should try to provide administrative workflows to ensure that procedures are in place for updating content regularly. Technology can help. In the past, it used to take FirstGov administrators one to two days to update content because the material had to go to the site manager, an outside company, for posting. To speed postings, OCSC recently implemented content management software that lets GSA staff members load content themselves.
“The most valuable real estate on our site is the front page above the fold,” Godwin says. “To take full advantage of that, we have to be able to update it in real time, especially for fast-breaking developments like Hurricane Katrina.”
Of course, even the most usable and up-to-date site is entirely useless if its visitors don’t speak English and the site only posts English-language content. Pizzella says that when the 2000 Census revealed that about 28 million people speak Spanish at home, OCSC quickly got to work creating a Spanish-language version of FirstGov. In fact, FirstGov’s Espanol.gov earned a 79 ACSI rating, seven points higher than its English language counterpart. “This is definitely something we’re encouraging other government agencies to do,” Pizzella says.
The Housing and Urban Development Department, the Small Business Administration and the National Library of Medicine all provide Spanish-language content. Social Security provides information in 15 different languages.
Although OCSC sees its mission to help other government entities create better-designed sites, it also tries to encourage them to expand the types of transactions users can perform on federal sites. Godwin hopes that more sites will move from being strictly informational to also providing online services.
“Since around 2003, we’ve seen more and more citizen demand for the ability to do things online, not just get information,” Godwin says. She chalks this up to the fact that citizens are becoming increasingly comfortable with commercial Web transactions. Deciding to offer online permit or grant processing admittedly also often requires policy or even legislative changes, but Godwin contends that the more such transactions an agency can offer on its site, the higher the user satisfaction level will be.
On the flip side, it’s also important not to eliminate other communications and information services or transaction processes, Pizzella adds. As FirstGov.gov expanded, OCSC officials opted against abandoning its information hotline. But the team did tie the telephone service to its Web effort by having the same customer-service people respond to e-mail originating at the site and to hotline queries. “We wanted to make sure that no matter how people reach us, they get the same response,” Pizzella says.
Another advantage of having the same team handle both e-mail and telephone queries is that the process tends to reduce workload peaks and valleys: When incoming telephone calls drop off, administrators can assign more people to handle e-mail and vice versa.
Pizzella also recommends that agencies do careful evaluations of paper brochures before eliminating them because some still have a place in the federal communications chain. “If someone needs first-aid advice, they’re not going to go to the Web to find it,” she points out. And while online Adobe Portable Document Format versions of brochures provide consumers with immediate access and save government printing and mailing costs, many citizens still prefer traditional printed and bound booklets. People with no Web access require them, and Web users may prefer booklets because they’re easier to find in a bookshelf and sometimes easier to read than printed PDFs.
Currently, OCSC is working on an upgrade to FirstGov. GSA has not unveiled all its plans for the new design, but it will offer new features such as podcasts and streaming video.
Godwin and Pizzella are upbeat about the state of government sites, although they admit that many agencies need to work to improve their sites’ usefulness and boost satisfaction ratings. “We’re still in a learning curve,” Pizzella says. “But we’re on an upward trend.”
A quick look at the Energy Department’s home page reveals DOE’s quite diverse mission. The Web site includes topics that range from the scientific, such as carbon sequestration and climate change, to the consumer-related, such as energy efficiency. The fact that visitors can see all this on the home page is a testament to the department’s efforts to make maximum use of Web usability techniques and organization.
Despite being blessed with a plethora of scientific papers and data and cursed with the need to organize them, Energy designed the site around one guiding principle, says Rocky Campione, a senior policy adviser to the CIO: “Get to most pages within three clicks.”
Campione also points to a second principle, although it’s not found in the specifications for the new design. He calls it “the grandpa test.”
“My grandpa is always yelling, ‘What is the government doing for me?’ ” Campione says. “We try to design the site so that people coming there can easily find what DOE is doing for them.”
For instance, the home page has a drop-down menu that lets users select their state to find local DOE projects.
The home page also has what it calls “quick clicks” — jump pages for four types of users: consumers, researchers, educators, and students and kids. There are mouse-over menus by topic area such as energy sources and the environment. And the center column of the three-column home page is reserved for news.
Fitting as much information as possible on a relatively small number of pages was a challenge, says Amanda Foust, a Web designer with GovTech Solutions. The Akron, Ohio, company designed the site for DOE. “We used every trick we could think of,” she says. These included Web browser functions such as the mouse-overs and converting many pages from one column to three columns. GovTech also puts synopses on a page and then provides click-throughs to the complete information.
A good part of the three-and-a-half-month design process involved culling out old, useless and outdated information. With the old design, Energy employees often would post new documents to the site with no thought to organization. The new design encourages better content management and requires CIO office approval,
Campione says.
But Campione says this is not to discourage people within DOE from publishing information online. He plans to offer training over the next year so that eventually DOE staff will be able to post content directly. Energy has installed an application to control layout via a set of templates for flowing documents. This enforces a standard look and feel. It also controls the placement of pages to maintain the three-clicks rule.
Although DOE didn’t revamp the site on a shoestring, the work didn’t bust any budgets either. Campione wouldn’t reveal exact numbers, but he says he was able to keep it within the CIO budget by postponing a few other projects. The tight budget kept DOE from hiring usability testers or running focus groups. Instead, the CIO staff used Energy employees to test the site. “We tried to get everyone to think like an outside user, not like an employee,” Campione says. Having such a diverse mission helped because most employees have only specific knowledge about one or a few programs, not the breadth of the department’s work.
The Look
In terms of layout and color scheme, the developers had two goals. First, they wanted to maintain the DOE branding used on non-Web documents. Second, “because it’s the Energy Department, we felt we’d try to convey the feeling of energy,” says GovTech president Randy Cole.
GovTech gave Campione five designs to choose from, and the team selected one that featured a blue banner with streaks of white light running through it.
Campione says that while the DOE did have to skimp on user testing, it put in the necessary time to check the running of the site before it went live. “We tested it for days, getting most of the kinks out,” he says.
“We have the information that people who come to our site need. And now they can find it, and they can do so in seconds rather than minutes,” Campione says.
improve navigation because it lets an agency document users’ experiences performing specific real-world tasks.
A common aspect of human nature hamstrings most Web
designers: It’s very easy to fall in love with one’s own work. Self-assessment is difficult.
To avoid the trap of thinking that a site that works well for the designer and internal people will be as useful for outsiders, many agencies turn to usability testing.
While some Web site developers test by asking employees to work with the site and give their opinions, usability experts say such internecine evaluations are unreliable. “Many of the people who do the testing may not be representative of the people the site is aimed at,” says Sanjay J. Koyani, senior usability specialist in the Health and Human Services Department’s Web Communications Division.
Some agencies use focus groups of volunteers, generally asking them to use a site for an hour or so and then convene to discuss their experiences. Although focus groups can provide general subjective feelings about a site, they usually can’t pinpoint specific problems, Koyani says.
In contrast, usability testing documents users’ experiences performing specific tasks on the site. It clearly indicates whether and how quickly people are able to navigate to their desired goals.
Task Masters
During usability testing, the test team asks volunteers, who should represent a cross section of potential site users, to perform specific tasks. The test team is in the room with the volunteers, while others sometimes watch from a remote location and document how easily the subjects find their way to the desired goals.
Ideally, usability testing is done early and often in the design process, long before the site goes live. Some tests even use paper mockups of pages, asking users to point to links they would click on and then handing the users the mockup of the screen on which they’d land.
Koyani, who recently completed usability tests of the HHS-managed pandemicflu.gov, agrees that in a perfect world, agencies would always test sites as part of an iterative design process. But he says because of the critical nature of the content, HHS rushed to launch the site, so he had to test a fully functioning site.
For the recent test, he asked subjects to perform the 15 most important things HHS hoped to convey. Two examples of test queries:
• “You want to know the difference between pandemic flu and avian flu. Where would you find information explaining the difference?”
• “You are worried that your family may not be prepared for a pandemic flu. You want to find information on what your family can do to prepare. Please find information on this site that would help you do this.”
For HHS tests, Koyani evaluates each subject’s ability to complete a task on a three-point scale:
0 Ended or not completed
1 Completed with difficulty or help
2 Easily completed (minor sidetracking)
HHS is now evaluating the test results for the pandemic flu site. What happens after a test varies greatly, depending on the results. If a test results in major design changes, then further testing might be warranted.
The ultimate goal of any test, Koyani says, is to provide easier navigation for users.
Additional Reporting by Vanessa Jo Roberts
