Challenges, Deconstructed

Credit Payouts

Completing Challenges are not done just for the fun of it, of course. There’s the payoff in credits, the core of Reach’s player investment system. I actually began this entire exercise because of the BWU I quoted at the beginning, which I’ll repeat here with an emphasis:

Earlier in the week, he sat me down and showed off the full matrix of daily and weekly challenges we have at our disposal. I have to say that there’s some decidedly difficult stuff packed inside. SLASO runs? Ho, boy.

Before Luke showed off his adults only tool-set, I’d thought the challenges were mostly set in stone. Not the case. Over time, we’ll be able to roll out additional challenges and scale the payouts for the existing set to track with trending overall player progress.

The bolded comment is what piqued my interest enough to start tracking the Challenges. I was interested to see the extent to which the payouts rose over time. While the rest of the investment system is beyond the scope of this article, it’s important to note one aspect of it. As players increase in rank, the credits required to reach the next rank up gradually increase. However, that rise is softened because players earn more and more credits for each game played; the Game Complete bonus is tied to rank and has a gradually rising cap for each. Low ranking players max out at a few hundred cR per game, while Nobles earn over 3,800 for a long game. So it made sense that the Challenge payouts would likewise rise as the overall player population advanced in average rank, in tandem. There’s only one problem: it hasn’t happened.

Below is a trend line showing the total credits paid out by Challenges for each week since launch. (Note: I’m excluding the launch week, because Reach’s Tuesday release means the first week was missing one day, or four Challenges; the weeks included are full seven day weeks only.)

The graph shows that over the first five weeks of Reach’s release, the total credits paid out rose steadily, before hitting a plateau at about 40,00 cR per week. On week 11, payouts jump, and have since been moving erratically in a rough range between 50,000 and 60,000 cR per week. Overall, payouts have been flat for all of 2011, and have actually declined over recent weeks, even factoring in the steadily rising payout for the weekly Challenge LASO runs.

This means that as the Reach player population has increased in rank, the Challenge payouts have become an increasingly small proportion of the credits players earn; they have not scaled as BWU note indicated they would, and the way the Game Complete bonus does. Four months ago, the 5,000 cR for a brutal, no-death L.D. Legendary run through a Campaign mission would have been like hitting the jackpot. But moderately high players can earn that much in a game or two of regular matchmaking, making the payout for such a steeply difficult Challenge seem token.

I won’t pretend to know why Challenges have not scaled over time, as was indicated, but it general there seems to be a lack of detail given to the management of cR payouts. While aggregate payouts have held in a fairly narrow range, the actual payouts for individual Challenges have been swinging wildly about. None are more illustrative than the payout for completing Nightfall on Legendary without dying:

Date Challenge cR
10/17/2010 Complete Nightfall on Legendary without dying. 4000
11/5/2010 Complete Nightfall on Legendary without dying. 1250
12/11/2010 Complete Nightfall on Legendary without dying. 5000
12/17/2010 Complete Nightfall on Legendary without dying. 5000
4/13/2011 Complete Nightfall on Legendary without dying. 1700

 

This is typical of many Challenges.

Challenge payouts could scale over time in any number of ways. The simplest would be to gradually ramp up the payout for an identical Challenge over time. Another would be to gradually roll out more and more difficult ones over time, and increase the cR associated with each. Or, a combination of the two. For Challenges such as the Nightfall one above, that is not happening, but rather the cR assigned appears almost random.

For many of the Challenges structured around repeated actions, such as kills, a pattern emerges similar to the overall credit payouts. The first several weeks show a gradual rise in credits, linked to both an increase in requirement, and payout for them. And then, after a few months, the pattern wears off and the credit rewards become erratic within a range. To illustrate with a typical example, below is a chronological listing of all of the Multiplayer Challenges to kill a certain number of players within a single game, the cR for each, and how many credits get paid out for each kill.

Date Challenge cR Instances cR/Instance
15-Sep-10 Earn at least 17 kills in a multiplayer Matchmaking game. 1200 17 71
17-Sep-10 Earn at least 12 kills in a multiplayer Matchmaking game. 800 12 67
8-Oct-10 Earn at least 23 kills in a multiplayer Matchmaking game. 1700 23 74
25-Oct-10 Earn at least 17 kills in a multiplayer Matchmaking game. 1200 17 71
23-Nov-10 Earn at least 15 kills in a multiplayer Matchmaking game. 3000 15 200
10-Dec-10 Earn at least 15 kills in a multiplayer Matchmaking game. 3000 15 200
28-Dec-10 Earn at least 17 kills in a multiplayer Matchmaking game. 3000 17 176
31-Dec-10 Earn at least 12 kills in a multiplayer Matchmaking game. 1600 12 133
28-Jan-11 Earn at least 15 kills in a multiplayer Matchmaking game. 3000 15 200
29-Jan-11 Earn at least 23 kills in a multiplayer Matchmaking game. 2300 23 100
11-Feb-11 Earn at least 15 kills in a multiplayer Matchmaking game. 3000 15 200
21-Feb-11 Earn at least 12 kills in a multiplayer Matchmaking game. 1300 12 108
22-Feb-11 Earn at least 21 kills in a multiplayer Matchmaking game. 2100 21 100
5-Mar-11 Earn at least 23 kills in a multiplayer Matchmaking game. 2300 23 100
20-Mar-11 Earn at least 15 kills in a multiplayer Matchmaking game. 1500 15 100
29-Mar-11 Earn at least 7 kills in a multiplayer Matchmaking game. 1000 7 143
1-Apr-11 Earn at least 19 kills in a multiplayer Matchmaking game. 1900 19 100
19-Apr-11 Earn at least 21 kills in a multiplayer Matchmaking game. 2100 21 100
22-Apr-11 Earn at least 10 kills in a multiplayer Matchmaking game. 1200 10 120

 

There is no pattern to how much the Challenge is worth, overall or on a per instance basis. Note that average payouts have dropped with the most recent appearances of the Challenge, a trend that follows what happened to overall payouts. This kind of inconsistency permeates nearly every Challenge category, with few displaying any link between the goals and the credits for achieving them. If the goal of a Challenge is to motivate players to try to achieve them, and the credits are meant to be part of that motivation, then they need to be managed more carefully than this.

The system could be much improved if the Challenges were reworked to incorporate a methodology to the reward system. The approach that I would like to see would simply scale up the payout as the requirements increased. For example, a more consistent system for kills in one game would look like this:

Challenge cR Instances cR/Instance
Earn at least 10 kills in a multiplayer Matchmaking game. 1000 10 100
Earn at least 15 kills in a multiplayer Matchmaking game. 1875 15 125
Earn at least 20 kills in a multiplayer Matchmaking game. 3000 20 150

 

This is the exact opposite of the actual payouts, where the per instance credits actually dropped recently, offering less and less reward as difficulty increased. The credits per kill were 143 cR for 7 kills, 120 cR for 10, and 100 cR for 19 (!!) kills in a game. When a Challenge becomes more and more difficult, but the payout drops, it actually demotivates me to try and achieve it; a steadily rising payout in sync with difficulty would make more sense.

Similarly, a steady methodology for mission completions in Campaign is needed, as there does not appear to be any system in place to match difficulty with a proportional payout. I won’t laden this article with any more tables of Challenges, but suffice it to say the Campaign completion ones bounce about a lot. Is a Heroic run with Mythic really half as difficult as Famine? Why is a mission on Normal with Thunderstorm worth just 500 cR less than Heroic with Cloud, Tough Luck, and Catch on?

A method to assign a base set of credits for difficulty with an adder for each skull, along with another for requirements such as no deaths, would bring some logic to the Campaign payouts.

The recent swing of the mighty ban hammer, this time aimed (to my delight) at Firefight idlers, has brought with it a delicious wave of salty tears. But it’s also an illustration for how seriously Bungie takes the reward system, and how they don’t want players ruining other people’s games to suck down credits. The community would be served well by someone at Bungie applying the same level of care and attention to how Challenges pay credits as they do to resetting the credits of idlers.

Conclusions

The Challenge system in Reach is a vast one, pulling together numerous gameplay systems across game modes. Pulling back from the individual Challenges reveals the huge range of game systems and statistics from which they were built.

From these variables, many specific to each gameplay mode, nearly two hundred unique Challenge templates were hand crafted. The breadth they cover is tremendous, but perhaps because of that breadth there is a lack of depth and complexity to the individual Challenges. In the end, it is the inability for Challenges to re-imagined once Reach shipped that holds the system back.

One can imagine a system in which Challenges are not baked into Reach, but rather fed into the game through the online interface in much the same way as the Challenges are selected now. But rather than select a template to use, the system would string together the completion criteria in a more flexible manner, drawing upon the game systems. That way, if someone wanted to construct a Challenge to kill two Hunters in Firefight on Legendary, or to finish ONI with no vehicle kills and no deaths on Heroic with Catch on, they could. (To build a few random examples.) That would obviously be a much more complex and difficult approach to implement, but it seems like the natural direction for Challenges to evolve in. The cornerstone of a long-term support model will always be limited if the structure it’s built up on is fixed in the shipping game, rather than flexible.

Bungie has a history of implementing imperfect ideas in one game that get dramatically improved in the next, whether in game features like Scarabs in Halo 2 and 3, or Forge’s UI overhaul from Halo 3 to Reach, to the limited Firefight mode in ODST, now bursting with features in Reach. Reach’s Challenge system feels like one of those first generation features, a largely successful endeavor that, if reworked for a second go round, could become something really incredible.

I find Challenges to be the most compelling aspect of Reach’s overall player investment system, and one which I hope is expanded upon in Bungie’s next title with the kind of flexibility that lets the designers innovate with them long after the game comes out, rather than locking them into place once the code is pressed to disc. But even with those limitations, they are a welcome and fun way to nudge players toward meta-goals each day, a fitting compliment to Halo’s signature freeform combat. 

I’ll conclude with a handful of data points that I couldn’t find a way to weave into the main story. If anyone has questions about the Challenges I didn’t cover, or want a bit of information sliced a different way, leave a note in the comments. I’ll keep tabs and reply with answers, assuming I can find them.

  • Most commonly used Challenge: Shootin’ and Lootin’, which has appeared 51 times
  • Number of Challenges used only once: 24
  • Total daily Challenges evaluated: 920
  • Number of unique Challenge names: 174

GhaleonEB

Pages: 1 2 3 4 5

6 Comments

  1. May 27, 2011
    Reply

    Brilliantly written. There were many points brought up I felt like you were reading my mind. I won’t spend a lot of time breaking down my personal wishes for the challenge system as you have touched on most and I agree that this is a system that Bungie will likely expand upon in future games and will be one of the main draws. I would like to comment on the cR payouts section as it is such a huge factor in motivation for players.

    As a more casual gamer, I love challenges and often check the dailies in the morning before work thinking how fun/difficult some of them will be when I attempt them later in the day… and that’s fun for me. Thinking about different strategies and ideas on how to complete them efficiently. As I’ve climbed the ranks and time has passed since launch, I can’t tell you how disappointing it is to see 4 daily challenges worth 1000 cR or so apiece. Like you mentioned, higher ranking players can earn well above the grand total here in a single match of matchmaking (depending on length). So, unless it’s something truly unique and fun, where is the motivation? Today’s dailies are a perfect example. Used to I wanted to nail em all… but now, I think “Why bother?”

    If you look what increased weekend cR payouts (and super jackpots) do to otherwise low numbered playlists, you can see how easily players are motivated by cR payouts. Team Arena went from a couple hundred to several thousand players in the playlist with the introduction of these super jackpots. Players are heavily motivated by cR so why not extend these to challenges? As you noted, we are seeing the opposite trend occur than you would expect at this point in the games lifespan. I understand Reach will still get some new players and you don’t want them jumping 3 ranks with one challenge completion, but to take challenges that once paid out 4000 cR and drop it to 1/4 of that now doesn’t seem sensible.

    I really hope Bungie (or 343i) takes note of what you’ve done here and has intentions of expanding on (and correcting) this system. It has huge potential as it is now even with it’s limitations.

    Excellent work.

  2. RC Master
    May 27, 2011
    Reply

    Good stuff. I’ve been wanting to see an analysis like this for some time and have considered doing it myself at some points!

    How did you actually collate the data and made tables? In an excel file? Database or some other method? Would you be willing to share that file? I’d love to be able to just plug in some queries and get some stats back out.

    On the article, you made a lot of valid points that I agree with wholeheartedly, and a couple of points that I think weren’t stressed enough. Overall you could have been much more critical (I certainly am :P)

    Multiplayer has a much higher susceptibility for play to being skewed in a negative way by external rewards. The kill, assist and other challenges still do this. You’re playing slayer and theres a kill challenge on? Gung-ho!
    Playing flag? Screw the flag lets just dong on the enemy team!
    Playing Grifball? Spawn-camping multi-kills ftw!

    The inherent per-action credits for kills does this already but offering challenges on top of this doesn’t help.

    I find myself consciously reaching this conclusion when a ‘get X amount of kills in MM’ comes up: “Hmmm. Multi-Team or Grifball – ignore the objective and kill dudes!”

    Challenges (and the credit system in general) should take into account the specific requirements of each game mode. At the very basic level by having one requirement for slayer and a different one for objective (as a parrallel to how kills in COD:MW1/2 were worth half as much in non-deathmatch.

    The feeling of kills > all else is so pervasive in Reach that many people literally do not realise that you get 20% bonus of your game complete when you win!

    I think its safe to argue that while multiplayer has a fair amount of replayability in the opponents, campaign practically demands that you should try wildly different, new and obscure things within its set framework since the encounters and enemies will not change.

    As noted there are a couple of examples of good challenges like this but they are too few and far between.

    The notion that there shouldn’t or couldn’t be any weapon or medal specific challenges in campaign is plainly ridiculous when you consider how most kill challenges in the mode would be got: by grinding on the same enemy or few enemies.

    Perfect example: http://halo.bungie.org/news.html?item=31730

    At the very least a challenge that was ‘assassinations’ or ‘super-combies’ would mix things up a bit.

    The point about challenge payouts failing to scale well is spot on and I love that I can point to that graphic now. If they’re not tracking the average population rank (mean, mode, median – something!!) they damn well should be. Or if they are I can’t believe that the population has ended up so static over the last several months!

    The no-death challenges for 5K are a good example : when that challenge first came up I could afford to spend a fair amount of time on it and still feel like it was a good reward. Now if I’m going to spend more than 20mins on it there is no point as I could either be earning more elsewhere or doing something that’s actually awesome for its own sake (like trying to land a no-death single segment or improve a speed run).

    Of course, possibly the best solution would be to simply scale all challenge rewards by a player’s current level. So they can at least retain the same relevance to all players even if the reward/difficulty is still wildly off.

    Finally, don’t be shy about posting stuff like this; I enjoyed reading it. You can count on at least one reader for any further articles like this. 🙂

    Woah, ok, thats pretty long now.

  3. May 27, 2011
    Reply

    Thanks for the feedback guys, you both make great points.

    RC Master, you’re right about the MP Challenges. At one point I started to mention some of those issues – using Grifball or other game types to work on kills; Multi-Team King of the Hill on a day when there is Challenge for getting X kills in a single game is amazing. I pity the fools that actually wander into the hill. In the end I scaled back some of that stuff because it was turning into a list of complaints. I settled on using one or two examples to try and speak to the overall isues. But yeah, there are many more.

    For the analysis I copied the tables into Excel, and added some columns to tag them. From the website I got the Challenge date, name and description and cR payout. From there I tagged them by day of the week, game mode, and then objective (kill, assits, etc), three fields for secondary criteria (in one game, etc.). Then spent a lot of time playing with filters and pivot tables.

    I’ll see if I can put the file up into something like Google Docs so it’s available for all. I work in finance and play with spreadsheets all day, and probably get way too much enjoyment out of it.

  4. Lawnmower172
    May 27, 2011
    Reply

    Wonderful article and analysis, thanks for taking the time to do this! I would like to slightly change topic and add on another aspect of the cr system. I love the campaign, it’s my favorite game mode. But since I’ve moved my commendations up to silver and beyond, I don’t earn jack squat for the time invested (currently around 260 cr per mission). This makes it almost impossible to rank up or buy anything in the armory through campaign.

    I understand there was a very good reason Bungie capped the commendation cr in campaign. But how about fixing the mission complete payout to reward players who continue to play campaign, instead of the occasional daily ‘mission complete’ or weekly LASO challenges? This would give players who want to immerse themselves in the campaign story more reward to doing so, rather than just rewarding them for the occasional challenge.

    P.S. Thanks to HBO for linking to this article!

  5. May 30, 2011
    Reply

    Thank you for a well written article.

    I was new to Halo when I bought Reach, and the Challenge System made me want more!

    You’ve covered the points about game play, mode, count and type distribution very well. I feel that I will reference this article whenever I am asked about it to newer player who are seeing the light.

  6. Homeboyd903
    May 31, 2011
    Reply

    Hey, I was just wondering… as sort of a ‘wish list’ can you touch a bit on how you would like to see this system expanded upon in future Halo (or any shooter) titles? I know you’ve explained in some detail about how you’d like to see the current system expanded upon (within it’s current limitations), but I just wondered if you had some things in your head that would be brand new to this type of system that you personally would like to see added. I know I’m asking a mouthful but you never know where these guys can draw their inspiration from.

Leave a Reply

Your email address will not be published. Required fields are marked *