Interview with Chris Bryant, Inside Source at Warhammer Online
Chris Bryant, stalwart buy forums warrior and QA Lead, took some time off from squashing bugs to discuss with us what he does evrey day. You will want to read on, as Chris guides us through all the inner working of Mythic’s QA Department, and talks to us about everything from the different QA teams, to how new content is tested and more importantly, how bugs are fixed! These tips below could help you improve your warhammer accounts you acquire with us. Read below for in depth interview with Chris Bryant
THE INTERVIEW
Hey there, everyone! My name is Chris Bryant (or as most of the staff in QA calls me, Old Man Bryant, what do these youngsters know) and I am the Senior Lead for the Quality Assurance (QA) department here at Mythic. I have been with Mythic over 6+ years now and I have spent almost all of that time working QA on either Dark Age or WAR. So, if there is a feature that you love (or hate, which I hope not) within either of those games I was probably involved in the testing of the feature at some point. Being in Quality Assurance, I have to live by the following motto:
I don’t make it, I don’t break it, and I don’t fix it. I just get to tell other people their stuff is broken so it doesn’t affect their warhammer account.
It is really pretty rare these days for people to find a job that they really love. A job that doesn’t cause you to wake up in the morning and want to call in sick “just because,” you can’t stand to go to work. For me, this is one of them. I have actually had, on more than one occasion, my boss come up to me to yell at me to take time off because I can no longer accrue any more paid time off ( a sure sign that I love my job). It’s not that I don’t really love my free time, but coming to work is fun and challenging just about every day. When you come to work and still have a great time, why miss a day? Sometimes I still can’t believe they pay me for this.
When the Community guys asked me to write up a Dev Diary on the Day in the Life of the QA team, all I could think of was, “This is going to be really short.” How much space would I need to write, “We test stuff”? Seriously, I mean come on.
If you have seen Grandma’s Boy then you pretty much have a clue (albeit a very oversimplified one) about what we do here in the Quality Assurance Department on a daily basis. But unlike the movie, or even other QA departments throughout the industry, it’s different here. We don’t grind out game after game. We have a seriously focused QA staff here, some of us that have been on the WAR project since the first art zone was uploaded onto a server. That is a great advantage to the individual gamer because we become specialists, and are so familiar with the coding. Having dedicated testers on one project for that amount of time is pretty unheard of and unique in the industry.
How do we test the new content that is added to the game?
We use a couple teams within the Quality Assurance Department. For instance, we have “Embedded Testers.” These are the guys that sit in with their respective teams (e.g. User Interface, Combat & Careers, RVR, Content, Items, Art, International Engineering and Core Tech) and get very hands on. These people are the merely the first line of testing. Yet they spend a great deal of time testing their respective areas on our internal servers. Most of the time, this testing is done before a lot of their particular features are even in game. We want to be sure everything is going to run smoothly.
As an example, the Combat & Careers testers will be testing abilities on one of their Development “sandbox” servers with no spell effects and no animations and no feedback in the combat log to let them know what is even going on. They have to use all the same tools that their Devs use to sort out the basic functionality of the abilities. Many times the Content embedded folks have it even harder. Imagine your trying to test out entire new zones for new content that have no art—the quest objects and monster art is all placeholder and there are no waypoints. (I applaud those people because content is the one area I really personally hate to test.)
Just about every new system or content area in the game that makes it to the Live servers are tested by their respective embedded team. After these systems pass their initial testing by the Embedded crews, the issues are sent down to the Internal Core Testers (the Core Testing group is my group by the way), not to be confused with the player group of Core Testers, or Oracles as they’re called on the forums.
The job of the Internal Core Testing team is, essentially, to retest everything that is passed down to them from our Embedded groups. These folks are not specialized testers that spend their Mythic lives testing one particular area. The crew on the Internal Core Testing team gets their hands dirty in every aspect of the game. They are my “Go To” guys. If I have any questions on almost any system in our game they have an answer or can get one to me quickly. They really know their stuff and I lean on these guys everyday because I don’t actually get to get in there and get my hands dirty much anymore. For those of you that know me from my postings on our Bug Forums, they are the ones that test about 90% of the stuff to which I respond (the other 10% of the testing is done by our phenomenal Embedded groups).
So our Core testers and Embedded Crew don’t have to just test every new system that is implemented, but they all have to be tested on multiple servers. Then, as they work their way up the chain, until they reach the Live servers in either a scheduled patch or a Hotfix. So when a test request comes down to us to test the 1,000 new items that are going into the game, we have to test them on multiple servers (/sarcasm on) and boy is that a lot of fun (/sarcasm off). So, pretty much every new system is tested by multiple people on multiple servers before it ever reaches the Live servers.
So why do bugs make it up to the Live Servers?
The first answer is just the quick and easy one: we are human and we miss stuff. There are only so many variations of Ability X, when used with Ability Y in Zone A while grouped with Class B that we are able to check before we sign off on a new feature. (Feel like your back in trigonometry class yet?) I mean honestly, who knew that a small change to a User Interface functionality issue would break an ability that was working perfectly fine before the “fix.” And who knew that fixing that ability to work again would break another ability that is totally unrelated to the first … and so on and so on. We are usually under the most aggressive type of testing schedule and there are just some things, unfortunately, that we will miss from time to time.
What happens to that bug I reported in-game/on the forums?
The short answer: If you report it, it is tested.
Unless you are the 100th person to report the same bug, consider your particular issue tested. And even if you are the 100th person to submit the same issue, while we may not test the issue you reported, we did test one of the previous 99 that were reported. We test every issue our player-base reports to us whether it’s from our forums or the in-game appeal system. While you will never get a response from using the in-game Bug Report system, please don’t hesitate to use it. Lots of valuable information is pulled from the game and added to the Bug ticket that is generated by your report. Art issues in a particular area of a zone are one of those issues that if you don’t report it in-game, we probably won’t find it. Those in-game generated reports have items like your location in game automatically appended to the ticket.
Every morning we get a report from our Customer Service team about the most appealed issues over the previous day. This is the first thing our testers work on in the morning when they get in the door. While they are working on those issues from the CS report, I am combing through our bug forums and assigning out each and every new issue to be investigated.
While I am working on the forums, we have our Live team going through all the bugs that have been reported in-game and getting those into our tester’s hands. Once all of the issues that have been reported from the previous day have been assigned out to the testers, their day of testing begins.
So between testing all of the new content that will be going into the game, and all of the reported bugs from the player base, our tester’s days are pretty much full from the time they walk in the door until they leave each night. Hopefully this gives you all a little insight into the workings of our Quality Assurance department and removes some of the mystery of what happens when you report a bug.