Please click here if you do not see a Nav Frame at left

LESSON #5:

Testers -- The Unsung Heroes of Games

NOTE: these articles are primarily aimed at aspiring game designers, but many of the concepts described herein also apply to those who aspire to other types of jobs in the game industry. This article is subject to changes and improvements; reader comments are welcome. That goes for the previous article too, not just this one.

You have undoubtedly heard that a recommended way of getting started in the games biz is to get a job as a game tester. That's true, especially if you do not have a programming degree, an art degree, a business degree, etc. and if you can get the testing job with a game publisher or developer (rather than at a game testing lab located far away from game publishers or developers).

And you have undoubtedly also heard a lot of negative reactions to this advice. A lot of the negative things you have heard are probably from losers who couldn't hack it even as testers, or from guys who just approached it the wrong way. There is a common perception that testing is a "lowly entry-level job" and that testers are at the bottom of the totem pole. The fact is, testing is extremely important and the test phase is vital in polishing a game into a fun experience for the end user.

That's not to say that if you have an art degree or a programming degree, a law degree, or a business degree (or even a "game design" degree, if you find a college that offers such), that you MUST begin in the game industry in Q.A. Obviously, if you can enter the industry in a job closely related to the subject you mastered in at college, then you should target that path instead of Q.A.

But for those who have not gotten a degree in one of those areas... Testing can be an excellent way to get your foot in the door, for a lot of reasons that will be explored in this article.

Terminology note: The Test department of a game publishing company is called "Quality Assurance," or "QA" for short. The term "QA" is also often used to describe the function or process of testing. In this article, the terms "test" and "QA" are sometimes used interchangeably.

Also note: This article is discussing the full-time internal job of tester (wherein the tester is an employee who comes to work daily at the game company to test games, for wages). Volunteer (unpaid) "beta testing" (wherein someone at home gets a copy of a game and provides feedback via email, usually without pay) is a separate matter entirely. Getting a job as a tester can be a good way to get started in the game biz -- volunteering to do some beta testing is more akin to simply being a customer (an end user).

Quality Assurance testing jobs are usually found at game publishing companies. Developers also do some testing (but usually not full Q.A.) - but game development companies probably don't have full-time testing jobs (unless the development company is very large and well-staffed).

Typically, someone who works at a small game development company usually performs multiple job functions. Publisher jobs are usually more specialized. So someone who starts as a tester at a publishing company might eventually move up into producing, while someone who works as a tester at a smaller development company might eventually move up into any of a number of roles.

There are also independent testing labs who hire testers. Publishers are increasingly outsourcing their Q.A. to these outside labs. Jobs at these places are okay if you just want to test and you don't have aspirations of moving up in the industry - a tester who works at one of these labs would have to quit in order to move up in the industry. It's recommended that if you want to work as a tester as a steppingstone to other jobs in the industry, that you work for a publisher or a large developer, not an independent test lab. If you do not understand the difference between a publisher and a developer, see FAQ 28. Working at an independent Q.A. company is not as good for building a game industry resume, unfortunately - you don't get to interact with developers and producers as much, and without exposure to the daily goings-on of a developer or publisher, it's harder to move up into the industry from a test lab. The remainder of this article is based primarily on working in the Q.A. department of a publisher.

If you want to volunteer as a beta tester, try hanging around at http://www.bluesnews.com and watching for announcements of public betas. I make no guarantees that you can get taken on if and when you respond to such announcements, but if you do, it might be helpful in getting a tester job later on (if you do an excellent job as a beta tester).

TESTING -- THE FINAL PHASE

In a large game publishing company such as Activision, the QA phase comes towards the end of the project. And the testers are usually not involved in the project until after most of the work on the game has already been performed. This can have some unfortunate consequences, since testers brought in at the end were not involved in design decisions and don't necessarily know the rationale behind them. In a smaller company, team members who helped create the game may put on their tester hats towards the end -- thus they are already aware of the circumstances that led to project decisions made along the way.

The fact that most testers come in at the end of the project, powerless to have a major impact on the design of the game, is perhaps what leads to some of the negativity about the job. A military analogy can be drawn, putting the testers in the role of foot soldiers and the design/production team as the officers in a battle. This analogy has a limited usefulness, so I think it's worth mentioning, but this analogy falls apart if you try to apply it across the board to the process of making a game.

The foot soldier does not have the general's-eye-view of the battle, and is expected to just do what he's ordered to do. In a battle, there's rarely time to pass the strategic thinking all along the line so every foot soldier understands what the general is doing. In the process of making a game, I like to share my strategic thinking with my testers as much as possible -- not all designer/producers do things the way that I do. It's a hard thing for the testers to have to accept that it's too late to add features, and I'm sympathetic to that.

TESTING IS REALLY IMPORTANT

As a designer and producer, it's very important to me that the games I make are easy to use, friendly, and fun. I am my game's worst critic. In the QA phase I typically am the most prolific writer of bugs. But I'm also the guy who often has to reject testers' bugs as "Not a bug" or "Works As Designed." Sometimes the tester whose bug is rejected may think I'm not on his side, but there are no sides! QA and I share the same goal -- to make a game that will be a positive experience for the end user.

The way to make sure the game will be a positive experience is through thorough testing. Get multiple pairs of eyes looking at the game, get multiple pairs of hands taking the game through all its paces. I play the game one way, but somebody else plays it another way, trying things I never thought of. So I need the help so all the flaws can be found before my game goes out the door.

TESTING AIN'T EASY

To some folks, the term "testing" conjures up a mental image of guys (and some gals) sitting and playing games all day. Sounds like fun, easy work.

Far from it. It can be fun in the long run (it's an enjoyable job as jobs go), but it often feels like work. And it definitely isn't easy.

It's fun to play a game, but it's not easy. Ask any gamer. If a game is easy, it's no fun.

If a game isn't fun yet (if the balance isn't there yet, as sometimes happens), then it's definitely not easy to force yourself to continue playing it. If your job is to test, you have to do it even if it's no fun. And THAT ain't easy!

And when you're testing a game repeatedly, playing the same section over and over again, the fun of playing it can sometimes wear thin.

Testers have to be computer literate. You may be called on to test a computer game, so you have to be able to install cards and joysticks and drivers, you have to be able to uninstall and reinstall operating systems. Even if you are testing Playstation 2 games or Game Boy Advance games, you still have to use a computer to write your reports.

Testers have to be good communicators. You can be the most awesome bug-finder in the company, but if you can't explain to the team how you found the bug, what the bug is, what should have happened versus what did happen, and what you think is going wrong down in the code, then it can't be fixed! This part is extremely important, so it bears repetition. A tester must type in complete sentences. A tester must understand, and habitually use, proper punctuation and capitalization. You cannot become a tester at a game company where everybody uses English, if you cannot communicate properly in written English. This is such an important requirement that I will repeat it another time, at the end of this article (FAQ) (Lesson).

A good tester doesn't just find a bug and report "I picked up the green sword and drank the blue potion, and the game crashed." A good tester digs deeper, figures out how to make it happen again, and, knowing how the game works, figures out what's really going wrong. Maybe if he goes back, picks up the green sword and drinks the blue potion again, the game won't crash. Maybe the circumstances that caused the crash are deeper than that. A good tester is like a bulldog (see? So much for the "grunts and generals" analogy!) -- tenaciously digging his teeth into a bug and not letting go until he figures it out (look there, even the bulldog analogy falls apart if you try to take it too far).

Testing is definitely not a job for someone looking for a fun, easy experience.

LEARNING THE BIZ FROM THE TRENCHES

Working as a tester is an excellent way to learn the game biz. Testers are exposed to (if not directly involved in) several other aspects of the business of games. Through working as a tester on several game projects, a lot can be learned about the business as a whole.

Development aspects. (Terminology note: "development" is used herein to refer to the "pre-production" aspects of a project, thus this refers to the design. "Production" is used to refer to the creation of the code, art, and audio.)
Through testing a game, the tester will have questions like, "why was it designed this way?" There are often several different ways that an interface for a particular feature could work. Sometimes those different ways are enabled as user-selectable features, and sometimes the designer just picks one standard way for the feature to work. The tester is among the earliest users for the game, so is often exposed to design aspects that subsequently get changed (or do not get changed).

Production aspects.
The tester finds a bug and reports it. Most of the time it's a simple matter of making a fix. But sometimes it's a complicated matter involving discussions between QA and the Production team. In the meeting at which the discussion takes place, the tester will gain insights into what it's like working on the Production team. After several such meetings, the tester gains a broader perspective on the process as a whole.

Marketing / Sales aspects.
Testers don't only test the game, they are also often called on to check the text on the package and in the manual. The hardware specification must be accurate, and so must the product claims (the bullet points describing the game's features). Through analyzing the product claims, the tester can gain an insight into how the product is being presented to the buying public.
Ship dates are hateful deadlines that often preclude the fixing of a pet bug. (More on prioritization of bugs below.) Through immersion in a few game projects, the tester comes to learn the importance of prioritizing bugs. As Dr. Laura says, "is this a hill you want to die on?" The company needs to ship its games in order to make money to pay the testers. Games can't be tested and fixed forever, and as a tester is involved in the process a few times, he learns about the realities of making games for profit -- how to prioritize the bugs he finds.

Customer Support aspects.
No matter how thoroughly the testers pound on a game, there are bound to be some problems that only surface when the game is released into the wild. They're animals out there! They do all kinds of things to games that those in Production and QA never thought of, and that results in calls to CS (Customer Support). When those calls start coming in, the first call CS makes is to QA. The QA-CS relationship is therefore important. The tester who thought he was finished with that game is ordered back into service -- "try the game on this hardware configuration, or try doing this and that and see what happens." And the dreaded, "How could you have missed that?" The tester cannot help but learn about the kinds of issues CS faces.

Manufacturing aspects.
Even manufacturing is something the tester will learn about through his involvement in making a game.
- When the tester is given a box and manual to approve before the game is finished, the tester will learn that it takes longer to print a box and manual than to run off the CDs.
- When the game is a console disc (rather than a computer disc), the tester is exposed to the issues involved with platform manufacturer approvals. And to the fact that patches are not possible. It has to be right the first time with a console disc. And on those rare occasions when something goes wrong in the manufacturing process, the tester may be impacted by having to re-test and re-release. And even if it doesn't have to be re-tested, through the day-to-day immersion in the culture, the tester learns all the painful details of what happened with that finished game before the tester gets his own copy (which he may not even want to play any more).
- There can even be differences between a manufactured CD and a gold CD burned internally (in a CD-R). On one of my computer games, we had made our music tracks the normal way according to "redbook standard," which worked fine when we tested them. Little did we know that when the game was sent off to be manufactured, that the CD manufacturer would put shorter blank spaces between the music tracks. When the manufactured game was played, the audio would be played from one music track -- but then, before the CD head went back to play the track again from the beginning, it would wander into the beginning of the next track (so there would be a brief false start of another tune before recycling back to the beginning of the proper tune for that level). We had to remaster all the music with blank space at the front of each track. I know I learned something from that -- and I'm sure the testers did too!

See what I mean about how the tester gets to learn a lot about the biz? And here you thought testers just played games all day.

TESTING IS A STEPPINGSTONE ... for those who are cut out for bigger things

That's right, there was some qualifying fine print in that heading. Not everybody who gets hired as a tester is cut out for bigger things. If you work hard and well, display a good cooperative attitude, communicate well and effectively, it's likely that you will be able to grow into higher positions.

If you shine as a tester on a couple of projects, you will likely get promoted to lead tester.

If you shine as lead tester on several projects, you may get promoted to test manager. Or someone in the production studio may want you to join their team as a production coordinator or assistant producer or even junior designer.

Just showing up and doing your work isn't enough to warrant promotion. You have to be bright, and you have to shine brighter than most. There's nothing unfair about that (despite the grousing you might hear from some who never seem to rise to higher jobs).

I forget where I heard this saying, but it seemed apropos to this topic:

"Smart people learn how to use their skills. Happy people learn how to live with their shortcomings."

If you are good, you will find a testing job to be an excellent entry to the games biz.

A GLOSSARY OF SOME TERMS FREQUENTLY HEARD IN QA

'A' bug -- The 'A' bug is the very worst kind of bug. This type of bug can be summed up thusly: "It would be an unthinkably bad disaster if the game was released with this problem unfixed." Some examples:
   - the game crashes;
   - there is a virus in the game;
   - there are obvious spelling errors;
   - there are obvious graphical or audio problems;
   - a feature (in a menu or accessed by pressing a button) does not function;
   - there is no copyright language in the game anywhere;
   - the game is not fun to play.
Releasing a game with this sort of flaw would generate very bad public reaction and bad press, or there could be legal ramifications against the company.

'B' bug -- The 'B' bug is not quite as bad as the 'A' bug. It can be summed up thusly: "It would be unfortunate if the game was released with this problem unfixed, but the game is good regardless." In a pinch, if the company has a need to release the game and stop spending money testing and fixing it, and if Customer Support, Sales, Marketing, QA, and the executive staff all agree, the game may be released with minor flaws. For example:
   - bugs which do not ruin the experience of playing the game;
   - noticeable graphical or audio problems (especially if you know where to look for them);
   - highly desirable features were left out (and are not mentioned anywhere in the game).
'B' bugs will likely show up in press reviews of the game but are things that are probably hard (expensive; time-consuming) to fix. The playing public won't be happy with these problems, but the overall playing experience is not ruined by the existence of these problems.

'C' bug -- The 'C' bug can be summed up, "It would be nice to fix this problem." The tester may feel strongly about this problem s/he has identified, but when weighed against the company's larger need to release the game, the bug isn't that big a problem in the decision-makers' view. When push comes to shove, 'C' bugs may have to fall by the wayside (if they're hard to fix, that is -- a 'C' bug that's easy and quick to fix is likely to simply get fixed, unless the project is coming down to the wire).

'D' bug -- "It would be nice to add this feature." Especially when reported later in the test process, 'D' bugs are likely to remain unfixed.

"All bugs should be fixed." -- Ideally, of course, this is true. But some games are so big and complicated that the fixing would simply never end. And some testers are pickier than other as to what constitutes a bug that needs to be fixed. There have to be checks and balances in a game company (just as there are in a governing body).

"Alpha" -- The terms "Alpha" and "Beta" are defined differently by every company. Especially, developers' definitions of these terms may vary from publishers' definitions of these terms. Some developers may prefer to define Alpha as "code that demonstrates how the game will play." But most publishers (specifically a publisher's QA department) would prefer to define Alpha as "everything has been implemented in the game but there are bugs and the gameplay needs tweaking."

"Beta" -- Some developers may prefer to define Beta as "everything has been implemented in the game but there are bugs and the gameplay needs tweaking." But most publishers (or their QA departments) would prefer to define Beta as "everything has been implemented and as far as the developer knows, there are no bugs and the gameplay has been fully tweaked."

"Beta testing" -- Quality Assurance testing is a different thing from Beta testing. We usually use the term "beta tester" to refer to volunteers who test for free from their homes. Q.A., on the other hand, is a full-time position, a paid job. Beta testing is a good way to break into real testing. Look for opportunities to volunteer when you see that a game company is seeking beta testers (usually in an online game bulletin board or something - it's hard to seek out beta testing opportunities, and I can't tell you how to do it, you just have to be active in the game community's online forums). Do the beta testing well, and you might get offered a real testing job.

"Can Not Replicate." -- Sometimes a problem will happen to a tester but he can't provide steps to replicate the problem. If the programmer can't cause the problem to occur, with the debugger running to reveal the source of the problem, it may be difficult to fix. A good tester will try to make the problem happen again or figure out why it happened.

"Gold Master" -- The CD or DVD released by QA to manufacturing. This disc has been verified and virus-checked and has gone through an extensive checklist before it's sent out the door.

"It's a feature." -- The corollaries to this one are "Not a bug" and "Works as designed" (below). Sometimes what the tester expects the game to do, and what the game does instead, cause a bug report to be written. The bug report goes to the designer, who says "that's not a problem -- that's the way I designed it to work, and here's why it should remain as is ..." If the testers can present a convincing argument that the "feature" is counterintuitive or unfriendly, then perhaps it needs to be changed.

"Need more info (NMI)." -- This comment is likely scribbled on a bug report that doesn't tell the programmer enough information about how to replicate a bug, or why the tester feels that it is a bug.

"Not a bug (NAB)." -- See "It's a feature" (above).

"Psychotic user behavior." -- Term used to characterize a problem caused by unreasonable user input. For example: "The game crashes if you press F10, then Esc, 30 or 40 times in a row." No reasonable user would do this, and even if someone does do it, it would be unreasonable to fault the game for crashing under these circumstances. If the problem is hard to fix and the project is coming down to the wire, it may be simply written off.

"Release" -- QA signs off on the game and puts their stamp of approval on sending the game off to be manufactured.

"Ship it." -- This phrase is heard at the tail end of the test process, when the test team is starting to see the light at the end of the tunnel.
   - Tester: "I found a bug."
   - Lead tester: "What kind of bug?"
   - Tester: "It's just a 'C' bug, not a biggie. Psychotic user behavior."
   - Lead tester: "Ship it!"
Finished (tested, quality approved) games are shipped by Fedex or other courier service (or sometimes, if really important and timely, delivered by a member of the team) to the manufacturing facility. Manufactured product is shipped by truck to the stores. "Ship it" is the mantra used to seal the importance of cutting off testing and releasing the game into the wild.

"Tweak" -- Synonymous with "adjust."

"Will not fix (WNF)." -- When time is running short, and minor bugs are reported, the programmer or the designer or the producer may scribble this cryptic note on the bug report. All bugs have to be "closed" (resolved) before the game can be released.

"Works as designed (WAD)." -- See "It's a feature" (above).

WANT TO GET A JOB AS A TESTER? HERE'S HOW TO PREPARE

I recommend that you have a college degree before applying for a job as a tester, but it's possible to get a testing job without one. But consider for a moment -- what is your ultimate goal? If you eventually want to become a designer or producer or move up into marketing or become an executive, a college degree is definitely helpful. If you just want to be a tester (and do not have any goals beyond that), then fine, a high school diploma might suffice. But guess what three attributes or skills you need first and foremost to be a tester...?

Communication skills - The tester must be able to communicate in two ways: via the written word and via the spoken word.
Computer literacy. Testers know how to take computers apart and put them back together. Testers know how to browse the Internet, and they know all about email, instant messaging, and chat room netiquette. Testers know how to troubleshoot installation issues, download drivers, update virus DAT files, and upgrade computers. Testers know how to use word processors, imaging programs, scanners, and modems. Testers are often called upon to make screen shots of games, so you need to know how to grab a shot, and crop it in Photoshop or GIMP. Especially important: know how to use a database program. Check out Bugzilla and Mantis, fool around with them to create some sample bug reports.
Game literacy. Play as many games as you can. Compare the pros and cons of this game versus that game. Read game magazines. Know the difference between an FRP and an RTS. Online games, console games, handheld games, board games, CCGs.

Snap reading comprehension quiz: What are the three attributes needed for a game tester?
For extra credit: Can you think of any other ways to improve your skills in these three areas?

Location, Location, Location

If you want to get a job as a tester, you must live near a game company that employs testers. You have to be able to commute daily to the game company. You canNOT get a full-time job testing games from your own home.

Testing jobs are much more common at game publishers than at game developers. (Read FAQ 28 to learn the difference between a publisher and a developer.) And you can research game companies using my Game Biz Links page. If there are no game companies near you, move.

Some game companies who need testers use temp staffing agencies. They don't hire the testers themselves - rather, the temp agency hires the testers, and "loans" the testers to the game company. These agencies may call themselves something besides "temp agency," by the way. "Staffing agency," "technology personnel provider," whatever. When I was younger I worked through such an agency and called them "job shoppers." Usually, if you apply to the company directly, they'll tell you the name of the staffing agency they get their testers through. But game companies are notoriously tightlipped, meaning you might not hear back from them at all. So if you're trying to break in through Q.A., look not only at the game companies near you, but the staffing agencies as well. Find the technical staffers, the ones that supply IT workers and stuff.

For more information about how to get a job in the game biz, read FAQs 4, 24, & 27. Don't do any of the Stupid Wannabe Tricks like believing scam artists, or basing your life plans on your fantasies about the game biz.

About those "game tester" websites

Do NOT send money to sites that promise game testing gigs online, like http://www.becomegametester.com or http://gametesterguide.com/ or http://ugametester.com other sites like that (pretty much any URL with "game" and "tester" or "testing" in it). You can't trust the promises made by any of those sites. There's even one that warns you against testing scam sites, http://gametestingscams.com/ - this one is also a scam. (^_^) It's warning you to beware of scams, but it's all a lie too (and it is undoubtedly the brainchild of someone who owns one of the sites it's "warning" you against). These sites exist to take money from you, not to help you get a full-time testing job. No reputable game testing service should ask YOU for ANY money whatsoever. (If you had a good experience with one of those sites, tell me all about it and I'll change my stance, and I'll share the news with other game biz wannabes.) I wrote more about this sort of questionable website in FAQ 24 and in my July 2007 column on the IGDA website, "The Games Game" - the editor entitled this column "Summer Job Scams (July 2007)." The column is at http://www.igda.org/columns/gamesgame/. If you look for the July 2007 column in a later month, just click Archives to find all older columns.

But you don't have to believe me. Other game biz pros have discussed the topic of these test scam sites on GameDev.net. Check'em out:

Community � Forums � The Business of Game Development � What Testers Make (started 3/31/08)
http://www.gamedev.net/community/forums/topic.asp?topic_id=488864

Community � Forums � Help Wanted � Found an awesome website to become a tester! (closed 3/31/08)
http://www.gamedev.net/community/forums/topic.asp?topic_id=488665

And look at what these scam sites offer to people who'll post links to their sites, if some sucker pays the scammer money:
http://www.affiliatebling.com/gamer-testing-ground/

That last one might change their URL or close down when they find out that people are showing you (the potential sucker) about how they make money, so here's what it says on that site as of March 31, 2008:

Gamer Testing Ground
"Get Paid to Play Video Games."
Make Up $30.44 For Every Sale + Another $20.44 In Cross-Sales!

Let your visitors learn how to make money for playing video games.

Tiered Membership, 75% commission on each sale:

($34.95) � 6 Month Membership - your commission: $23.50
($39.95) 12 Month Membership - your commission: $26.96
($44.95) 24 Month Membership - your commission: $30.44

The target audience for this website would be people 18-30 years of age, as kids will not have a credit card and will waste your click-throughs or will request a refund when their parents find out that their credit card has been used without permission.

*NEW* Cross Sell Opportunity:

In the backend we keep track of which affiliate referred which customer. When a particular customer logs in to the members area he is presented with an opportunity to buy membership to Gain Opinion at a discount rate. To buy the Gain Opinion membership this customer will go through YOUR ClickBank ID.

Additional benefit: up to $20.00 extra commission from EACH member.

Do I need to say it again? Beware of websites that ask you to pay for a job as a game tester. Don't fall for the hype. They just want your money, that's all they're about.

In Conclusion

I've said it before, and I'll say it again. (Right now, as a matter of fact.) If you want to get into the game biz (either to design your own games or just because it is a field that's interesting to you), testing is a great way to get started. Don't listen to those naysayers who say testing is a dead-end job. Most people don't realize how hard testing is. Or how important it is in the process of making games.

For MORE about testing, see Lesson 17: MORE About Testing -- The View From Inside Q.A., by guest lecturer Matthew S Burns!

Additional reading: http://www.igda.org/Endeavors/Outreach/Students-Newbies/wisdom/wisdom_boutin.htm.

Want the latest word on salaries in the games biz? Go to gamecareerguide.com and type "salary survey" in the site's Search box.

Got a question about this lesson? No need to raise your hand -- just click here to go to the bulletin board. You'll get answers!

Click here to go to the previous lesson.
Click to go ahead to the next lesson.

Click here to return to the Sloperama Game Biz FAQs main page.

LOG:

Minor editing (paperclip exercise paragraph), April 22, 2001
Added definition of beta testing, August 13, 2002.
Added (in bold text) the part about how important writing skills are - in two places in this FAQ, Feb. 10, 2004.
Later edits are logged at sloperama.com/advice/log.htm.