Crowdsourcing for information visualization: Promises and pitfalls

Rita Borgo; Bongshin Lee; Benjamin Bach; Sara Fabrikant; Radu Jianu; Andreas Kerren; Stephen Kobourov; Fintan McGee; Luana Micallef; Tatiana von Landesberger; Katrin Ballweg; Stephan Diehl; Paolo Simonetto; Michelle Zhou

doi:10.1007/978-3-319-66435-4_5

Crowdsourcing for information visualization: Promises and pitfalls

Rita Borgo, Bongshin Lee, Benjamin Bach, Sara Fabrikant, Radu Jianu, Andreas Kerren, Stephen Kobourov, Fintan McGee, Luana Micallef, Tatiana von Landesberger, Katrin Ballweg, Stephan Diehl, Paolo Simonetto, Michelle Zhou

Department of Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Chapter

19 Citations (Scopus)

Abstract

Crowdsourcing offers great potential to overcome the limitations of controlled lab studies. To guide future designs of crowdsourcing-based studies for visualization, we review visualization research that has attempted to leverage crowdsourcing for empirical evaluations of visualizations. We discuss six core aspects for successful employment of crowdsourcing in empirical studies for visualization – participants, study design, study procedure, data, tasks, and metrics & measures. We then present four case studies, discussing potential mechanisms to overcome common pitfalls. This chapter will help the visualization community understand how to effectively and efficiently take advantage of the exciting potential crowdsourcing has to offer to support empirical visualization research.

Original language	English
Title of host publication	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Publisher	Springer Verlag
Pages	96-138
Number of pages	43
DOIs	https://doi.org/10.1007/978-3-319-66435-4_5
Publication status	Published - 2017

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	10264 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Bibliographical note

Publisher Copyright:
© 2017, Springer International Publishing AG.

All Science Journal Classification (ASJC) codes

Theoretical Computer Science
General Computer Science

Access to Document

10.1007/978-3-319-66435-4_5

Cite this

Borgo, R., Lee, B., Bach, B., Fabrikant, S., Jianu, R., Kerren, A., Kobourov, S., McGee, F., Micallef, L., von Landesberger, T., Ballweg, K., Diehl, S., Simonetto, P., & Zhou, M. (2017). Crowdsourcing for information visualization: Promises and pitfalls. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (pp. 96-138). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 10264 LNCS). Springer Verlag. https://doi.org/10.1007/978-3-319-66435-4_5

Borgo, Rita ; Lee, Bongshin ; Bach, Benjamin et al. / Crowdsourcing for information visualization : Promises and pitfalls. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Springer Verlag, 2017. pp. 96-138 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inbook{1a59963848664a34915bcf65fca8957a,

title = "Crowdsourcing for information visualization: Promises and pitfalls",

abstract = "Crowdsourcing offers great potential to overcome the limitations of controlled lab studies. To guide future designs of crowdsourcing-based studies for visualization, we review visualization research that has attempted to leverage crowdsourcing for empirical evaluations of visualizations. We discuss six core aspects for successful employment of crowdsourcing in empirical studies for visualization – participants, study design, study procedure, data, tasks, and metrics & measures. We then present four case studies, discussing potential mechanisms to overcome common pitfalls. This chapter will help the visualization community understand how to effectively and efficiently take advantage of the exciting potential crowdsourcing has to offer to support empirical visualization research.",

author = "Rita Borgo and Bongshin Lee and Benjamin Bach and Sara Fabrikant and Radu Jianu and Andreas Kerren and Stephen Kobourov and Fintan McGee and Luana Micallef and {von Landesberger}, Tatiana and Katrin Ballweg and Stephan Diehl and Paolo Simonetto and Michelle Zhou",

note = "Publisher Copyright: {\textcopyright} 2017, Springer International Publishing AG.",

year = "2017",

doi = "10.1007/978-3-319-66435-4_5",

language = "English",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Verlag",

pages = "96--138",

booktitle = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

address = "Germany",

}

Borgo, R, Lee, B, Bach, B, Fabrikant, S, Jianu, R, Kerren, A, Kobourov, S, McGee, F, Micallef, L, von Landesberger, T, Ballweg, K, Diehl, S, Simonetto, P & Zhou, M 2017, Crowdsourcing for information visualization: Promises and pitfalls. in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 10264 LNCS, Springer Verlag, pp. 96-138. https://doi.org/10.1007/978-3-319-66435-4_5

Crowdsourcing for information visualization: Promises and pitfalls. / Borgo, Rita; Lee, Bongshin; Bach, Benjamin et al.
Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Springer Verlag, 2017. p. 96-138 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 10264 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Chapter

TY - CHAP

T1 - Crowdsourcing for information visualization

T2 - Promises and pitfalls

AU - Borgo, Rita

AU - Lee, Bongshin

AU - Bach, Benjamin

AU - Fabrikant, Sara

AU - Jianu, Radu

AU - Kerren, Andreas

AU - Kobourov, Stephen

AU - McGee, Fintan

AU - Micallef, Luana

AU - von Landesberger, Tatiana

AU - Ballweg, Katrin

AU - Diehl, Stephan

AU - Simonetto, Paolo

AU - Zhou, Michelle

PY - 2017

Y1 - 2017

N2 - Crowdsourcing offers great potential to overcome the limitations of controlled lab studies. To guide future designs of crowdsourcing-based studies for visualization, we review visualization research that has attempted to leverage crowdsourcing for empirical evaluations of visualizations. We discuss six core aspects for successful employment of crowdsourcing in empirical studies for visualization – participants, study design, study procedure, data, tasks, and metrics & measures. We then present four case studies, discussing potential mechanisms to overcome common pitfalls. This chapter will help the visualization community understand how to effectively and efficiently take advantage of the exciting potential crowdsourcing has to offer to support empirical visualization research.

AB - Crowdsourcing offers great potential to overcome the limitations of controlled lab studies. To guide future designs of crowdsourcing-based studies for visualization, we review visualization research that has attempted to leverage crowdsourcing for empirical evaluations of visualizations. We discuss six core aspects for successful employment of crowdsourcing in empirical studies for visualization – participants, study design, study procedure, data, tasks, and metrics & measures. We then present four case studies, discussing potential mechanisms to overcome common pitfalls. This chapter will help the visualization community understand how to effectively and efficiently take advantage of the exciting potential crowdsourcing has to offer to support empirical visualization research.

UR - http://www.scopus.com/inward/record.url?scp=85031501419&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85031501419&partnerID=8YFLogxK

U2 - 10.1007/978-3-319-66435-4_5

DO - 10.1007/978-3-319-66435-4_5

M3 - Chapter

AN - SCOPUS:85031501419

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 96

EP - 138

BT - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

PB - Springer Verlag

ER -

Borgo R, Lee B, Bach B, Fabrikant S, Jianu R, Kerren A et al. Crowdsourcing for information visualization: Promises and pitfalls. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Springer Verlag. 2017. p. 96-138. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-319-66435-4_5

Crowdsourcing for information visualization: Promises and pitfalls

Abstract

Publication series

Bibliographical note

All Science Journal Classification (ASJC) codes

Access to Document

Other files and links

Fingerprint

Cite this