Note: this was last updated in September; there is problem some information that needs changed on here.

1) How does this work?
Each team’s points-per-bonus, or PPB, is compared with other teams’ PPBs in a manner that tries to adjust the difficulty of a question set.

Powers per game are currently tracked but are essentially used as a tiebreaker. Of course, many sets or tournaments don’t use powers, so that complicates things.

2) Are all tournaments treated the same?

No. Naturally, some questions sets end up being easier than others. I try to adjust for this by taking the difference between the set in question and a NAQT IS set and averaging those values for the teams I’m following for these rankings. I use those values to modify actual PPBs for each set, and then use those to rank teams. I refer to these modified PPBs as “adjusted points-per-bonus,” or “aPPB.”

3) What kind of quiz bowl tournaments do you use in your rankings?

Tournaments that use pyramidal questions and have stats that report PPB at a minimum.

4) Why isn’t my team ranked?

There are a few reasons.
a) I lack usable PPB stats for tournaments you’ve played. This is not to say 20/20 is a superior format, but it is the basis of these rankings.
b) I’m missing stats for the tournament(s) you’ve played. If you email them to me, I’d be happy to incorporate them. If stats are never released in a usable format, then I can’t incorporate them.
c) The tournament(s) you’ve competed in don’t fit other criteria pertaining to these rankings (2 teams playing each other at a time, academic questions, pyramidal questions)
d) Your team’s adjusted PPB values aren’t high enough to get ranked in the top X, where X is however many I currently rank.

5) What do you prioritize when ranking teams?

The most important thing is a team’s best aPPB. If two teams have similar aPPBs, then I look at other good aPPB performances, power rates, and difficulty of tournaments played.

If two teams have one similar aPPB, but team A has higher aPPBs at their other tournaments than team B, team A has an advantage.

Similarly, a team with more powers per game will have an advantage. Teams whose best aPPBs come on harder sets have a slight advantage over teams that played introductory sets.

6) Do you rank the same number of teams all season long?

No. I usually start out with a small number of teams, and expand as more teams play tournaments.

7) Our actual A-team was missing a player(s) at a tournament we played at. Will this affect our rankings?

No, as long as you play a tournament with your full A-team at some point. If you end up missing players all season long, there’s no real reliable way I can adjust for that as of now.

If what is actually your B/C/lower letter team plays under the label of your A-team, let me know and I can adjust the stats I use. If your A-team is actually split, where half the team shows up and plays with 2 players from the B-team, I’ll end up listing those under the A-team’s stats and given low importance.

8) Why don’t you consider head-to-head records or tournament order of finish?

I find head-to-head to be flawed because it emphasizes the results of a single match over hundreds of bonus parts and tens of games. Tournament order of finish is in the same boat; it can be very dependent on the format of the tournament, which can be heavily affected by quirks in scheduling, minor upsets, etc.

9) What about points per game (PPG)?

That’s too easily affected by the quality of opponents. A fairly strong team that plays in a very strong region could easily have a lower PPG than a team of approximately equal strength in a fairly weak region, simply because the latter team would be able to put up more points on their weaker foes.

PPB, by comparison, is less affected by the strength of opponents and should reflected a truer estimation of the strength of a team.

Also, the game format will have an impact on PPG. Tournaments with powers will tend to have higher PPGs; tournaments with negs can lower PPG. The presence of worksheets, team rounds, directed rounds, and so forth can also increase PPG over tournaments that don’t use them.

10) How often are these rankings updated?

As often as possible. Last season (2011-2012), I did it on a nearly weekly basis during the season when events were occurring. I hope that automating the ranking process more will allow me to maintain a similar schedule this season.

11) Is my team’s a perfect ranking of our ability’s related to everyone else that plays quiz bowl? How accurate are your rankings?

Absolutely not. Sometimes teams don’t get their best performance(s) on record for whatever reason. Some good teams don’t play on pyramidal questions (I think they should, but that’s a separate issue).

Here’s some statistics I did comparing the accuracy of my final set of rankings before nationals and their performance at the 2011 PACE NSC and 2011 NAQT HSNCT.

Correlation between NSC’s final placement & HSQBRank’s pre-nationals rankings: 0.880297
Correlation between NSC’s ranking of teams by PPG & HSQBRank’s pre-nationals rankings: 0.899155
Correlation between NSC’s ranking of teams by PPB & HSQBRank’s pre-nationals rankings: 0.862467

Correlation between HSNCT’s final placement (with ties intact) & HSQBRank’s pre-nationals rankings: 0.713037
Correlation between HSNCT’s final placement (with ties broken by PPG) & HSQBRank’s pre-nationals rankings: 0.763574
Correlation between HSNCT’s ranking of teams by PPG & HSQBRank’s pre-nationals rankings: 0.797302
Correlation between HSNCT’s ranking of teams by PPB & HSQBRank’s pre-nationals rankings: 0.718643

http://www.hsquizbowl.org/forums/viewtopic.php?p=232529#p232529