Chance-corrected agreement

Two raters, one 2x2: build any table by hand

Drag the four cells of A-vs-B. The sliders start on the worked example: 2 both-urgent, 1 and 1 disagreements, 16 both-not.

Drag a cell to change how many of the 20 tickets fall there. Raw agreement, chance agreement, and Cohen's kappa recompute live below.
both say urgent2
A urgent, B not1
A not, B urgent1
both say not urgent16

A vs B, n = 20

Scores

Raw agreement po0.00
Chance agreement pe0.00
Cohen's kappa0.00

Beyond two raters

Three raters, five tickets: Fleiss' kappa

Click any vote to flip it between urgent and not urgent. Per-item agreement, the category shares, and Fleiss' kappa update live.

item R1R2R3 urgentnot P_i
Mean P̄ (P_bar)
0.00
Chance P_e
0.00
Fleiss' kappa
0.00

Category shares this round: urgent 0%, not urgent 0%. Chance agreement P_e = share_urgent² + share_not². If everyone always agrees on everything, P_e can reach 1.0 and kappa is guarded to 0.00 instead of dividing by zero.