Fast Dawid-Skene
share
Many real world problems can now be effectively solved using supervised machine learning. A major roadblock is often the lack of an adequate quantity of labeled data for training. A possible solution is to assign the task of labeling data to a crowd, and then infer the true label using aggregation methods. A well-known approach for aggregation is the Dawid-Skene (DS) algorithm, which is based on the principle of Expectation-Maximization (EM). We propose a new simple, yet effective, EM-based algorithm, which can be interpreted as a 'hard' version of DS, that allows much faster convergence while maintaining similar accuracy in aggregation. We also show how the proposed method can be extended to settings when there are multiple labels as well as for online vote aggregation. Our experiments on standard vote aggregation datasets show a significant speedup in time taken for convergence - upto ∼8x over Dawid-Skene and ∼6x over other fast EM methods, at competitive accuracy performance.
READ FULL TEXT