Reliable Multiple-choice Iterative Algorithm for Crowdsourcing Systems


This research is about on developing algorithms and statistical methods (largely based on machine learning) for solving problems in crowdsourcing systems We designed an iterative learning algorithm to infer true answers for multiple choice crowdsourcing systems, which shows better performance than the simple majority voting and expectation-maximization(EM) approaches. This result was presented at the ACM SIGMETRICS 2015 conference.

  • Paper Publications:
    "Reliable Multiple-choice Iterative Algorithm for Crowdsourcing Systems" (pdf)
       (with Junyoung Kim, Donghyeon Lee, and Kyomin Jung)
         - ACM SIGMETRICS, June 2015.
          (ACM International Conference on Measurement and Modelling of Computer Systems)

  • code: C++11 (zip) _copyrights: Hyunmin Lee, Donghyeon Lee, Junyoung Kim

  • (other) papers review (in group meeting @snu milab)
    - "Approval voting and incentives in crowdsourcing", ICML 2015.
         by Shah, Nihar B, Zhou, Dengyong, and Peres, Yuval. (presentation pdf)
    - "Is approval voting optimal given approval votes?", NIPS 2015.
         by Procaccia, Ariel D and Shah, Nisarg. (presentation pdf)

  • Crowdsourcing Research Community (Link).

    Concentration : Jun. 2014 - Feb. 2016