LESSON #17:
MORE About Testing -- The View From Inside Q.A.

NOTE: these lessons are primarily aimed at aspiring game designers, but many of the concepts described herein also apply to those who aspire to other types of jobs in the game industry. This lesson is subject to changes and improvements; reader comments are welcome.

July, 2002



Please welcome guest lecturer Matthew S. Burns (mattb2@earthlink.net), who is currently working at Treyarch, an internal studio of Activision. Matthew is also an occasional contributor to the Game Design BB and has graciously offered to share his perspective in regards to testing as an entry path into the game industry, based on personal experience. Feel free to contact him if you have questions, but please keep in mind that he cannot personally hire you or hook you up with a job.
Tom is taking a seat with the rest of the class, and when he speaks up, he does so in red text.

More On Testing Games

In this article, I will go over some concepts and also some vocabulary associated with game testing. If you have taken Tom's advice and chosen to apply for the position of QA tester, knowing this stuff before you go in to interview will help build up your confidence and (hopefully) make a better impression on your interviewer.

There's a lot more to testing games than most people think. Approaching a game as a tester usually means looking at it from a different perspective than the normal consumer. For example, place a standard RPG in front of a gamer and he'll just want to finish it and have fun, but put it in front of the person in charge of testing and he might say: "hmm, let's make sure that every kind of attack can be used against every monster in the game." As you can see, playing the game to test it is not the same activity as playing the game to have fun!

Checklists

A checklist is usually a spreadsheet-style paper (or set of papers) that insures that all permutations of the game's functionality work as designed. To throw out a random example, you might see a checklist that looks like this:

White Magic Abilities

--------------------------------------------------
|         | Cure | Cura | Curaga | Esuna | Regen |
--------------------------------------------------
|Tidus    |      |      |        |       |       |
|Auron    |      |      |        |       |       |
|Khimari  |      |      |        |       |       |
|Yuna     |      |      |        |       |       |
|Rikku    |      |      |        |       |       |
|Wakka    |      |      |        |       |       |
|Lulu     |      |      |        |       |       |
--------------------------------------------------

If you are familiar with Final Fantasy X, you'll realize that this checklist tests some of the game's healing spells against the game's characters. So a tester, having been given this document, would then go into the game and see: Can Tidus cast Cure? Can Auron cast Cure? etc. Now, most normal players of the game wouldn't bother teaching Rikku the Cure spell (she's much handier with items than magic). But that's exactly why it has to be tested.

You'll notice that this is just a tiny example of all the possible things you might check in Final Fantasy X. After checking every spell, don't forget all the items, normal attacks, specials, skills, overdrives, aeons, Rikku's special "use" items, all of the "customize" attributes possible on weapons, and (gulp) all the spheres on the Sphere Grid. Big games are the biggest pains to test...

Playthrough

If you are the kind of gamer who has gotten beating games down to a science, you may be called on to do exactly that: beat the game in as little time as possible. Let me tell you from experience that this is not particularly fun, especially after the first time you do it. You know all the cinemas, all the plot twists, all the levels, all the enemies' weak points, nothing is new and everything is boring.

Compatability (PC)

The PC is a mishmash of different components from different companies, and out of ten PC gamers you're likely to find 10 different PCs. One might have an nVidia GeForce 2 MX and a SoundBlaster Live, another may be using an ATI Radeon 8500 with an Aureal Vortex, and so on. If you know your PC parts, you'll know how many possible combinations there are, and the idiosyncracies associated with each component's particular drivers. Compatability testing insures that the game runs on as many of the most common combinations of video card, sound card, CPU, etc., as possible.

Submission (Console)

Console games must be approved by the console manufacturer before they are released. Every PS2 game is checked and approved by Sony, every GameCube game by Nintendo, and every Xbox game by Microsoft. This prevents the game from having grossly objectionable content, and supposedly insures the games all meet a certain standard of quality (some people may beg to differ). While each manufacturer has very similar requirements, there are plenty of case-specific situations. For example, if you made a game called "Microsoft Sucks" and the object of the game is to blow up Microsoft, don't expect it to be approved for release on the Xbox (maybe Sony would let it slip by...)

Because submitting the game to Sony, Nintendo, or Microsoft costs money and time, the company tries their best to get it right the first time to have to avoid submitting it several times over. Thus, testers are sometimes called upon to check the game against the manufacturer's own standards to make sure the game will pass when it is finally submitted.

Want to know more about the job of tester? Post a question on the Game Design bulletin board.

Tom adds: It's worth mentioning a few other points that haven't been made yet here or in Lesson 5.

  • Testing work can be seasonal. Some companies tend to lay off testers around September or October after the Xmas crunch period is over. Work can pick up again around March, when companies are preparing demos for the yearly E3 show.

  • You may find that the perks and advancement opportunities are better at smaller game companies as opposed to the big ones. But Q.A. jobs tend to exist mainly at big publishing companies - not at small development companies.

  • Q.A. is a low-paying job compared to the Studio. Q.A. is frequently where you "pay your dues" to get into the Studio. Want the latest word on salaries in the games biz? Go to gamecareerguide.com and type "salary survey" in the site's Search box.

  • In addition, because testers get paid for overtime (as opposed to the salaried Studio employees who do not), Q.A. tends to be a bit more regimented. Lunch time and break time is monitored; what time you show up and what time you leave are also watched more closely than in the studio. It costs a lot of money to pay testers to test a game, and companies have to watch the expenditures.

  • QA Lead David Steiner of SEK (SpieleEntwicklungsKombinat GmbH - http://www.sek-ost.de) wrote me in December 2005 to point out: "according to IEEE Standard 12207, the QA process is "a process for providing adequate assurance that the software products and processes in the product life cycle conform to their specific requirements and adhere to their established plans". The focus is not on optimizing product quality or providing design input, but on verifying that the game meets its pre-defined requirements.Ideally, QA testers are asked to check whether the game works as designed by checking the software against a list of requirements and features. If the design documents or specifications are unclear or missing, you may encounter problems organizing the QA phase efficiently."

  • There are game testing labs - small companies whose sole purpose is the testing of games for game publishers whose Q.A. departments are overbooked. Working at a test lab is not a surefire path to getting a job at a game development or publishing company. You can move up into management in the test lab, and you can gain valuable experience. But to get a studio job, you'd need to quit the test lab and look elsewhere. It's one way to gain industry experience, and if you live near a test lab, you can use this route.

  • Make sure you read Lesson 4, all about how to apply for a game biz job (like Testing). In case it doesn't mention it in Lesson 4, you need to live near a game company in order to work there. If you want to test games for Acclaim, for example, you need to live within commuting distance of their Glen Cove, NY, headquarters* (yes, they have offices in other areas, but their Q.A. jobs most likely are located in the HQ office). Game companies do not pay relocation expenses for new testers.

  • If you haven't yet read Lesson 5, what are you waiting for?

* The portion about Acclaim was written in 2002 - before the company shut down and auctioned off its assets. Don't bother looking for a job at Acclaim in Glen Cove as of December 2005!

Got a question about this lesson? No need to raise your hand -- just click here to go to the bulletin board. You'll get answers!

Click here to go to the previous lesson.

Click here to proceed to the next lesson.

Click here to return to the School-A-Rama main page.

© 2002 Tom Sloper and Matthew S. Burns. All rights reserved. May not be re-published without written permission of the authors.