Metadata vs. Ground-truth: A Myth behind the Evolution of Community Detection Methods

Tanmoy Chakraborty; Zhe Cui; Noseong Park

doi:10.1145/3184558.3186921

Metadata vs. Ground-truth: A Myth behind the Evolution of Community Detection Methods

Tanmoy Chakraborty, Zhe Cui, Noseong Park

Graduate School

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

6 Citations (Scopus)

Abstract

A community detection (CD) method is usually evaluated by what extent it is able to discover the 'ground-truth' community structure of a network. A certain 'node-centric metadata' is used to define the ground-truth partition. However, nodes in real networks often have multiple metadata types (e.g., occupation, location); each can potentially form a ground-truth partition. Our experiment with 10 CD methods on 5 datasets (having multiple metadata-based ground-truth partitions) show that the metadata-based evaluation is misleading because there is no single CD method that can outperform others by detecting all types of metadata-based partitions. We further show that the community structure obtained from the CD methods is usually topologically stronger than any metadata-based partitions. Finally, we suggest a new task-based evaluation framework for CD methods and show that a certain type of CD methods is useful for a certain type of task.

Original language	English
Title of host publication	The Web Conference 2018 - Companion of the World Wide Web Conference, WWW 2018
Publisher	Association for Computing Machinery, Inc
Pages	45-46
Number of pages	2
ISBN (Electronic)	9781450356404
DOIs	https://doi.org/10.1145/3184558.3186921
Publication status	Published - 2018 Apr 23
Event	27th International World Wide Web, WWW 2018 - Lyon, France Duration: 2018 Apr 23 → 2018 Apr 27

Publication series

Name	The Web Conference 2018 - Companion of the World Wide Web Conference, WWW 2018

Conference

Conference	27th International World Wide Web, WWW 2018
Country/Territory	France
City	Lyon
Period	18/4/23 → 18/4/27

Bibliographical note

Funding Information:
ACKNOWLEDGMENTS The work was partially supported by the Ramanujan Fellowship, SERB-DST, Govt. of India. REFERENCES

Publisher Copyright:
© 2018 IW3C2 (International World Wide Web Conference Committee), published under Creative Commons CC BY 4.0 License.

All Science Journal Classification (ASJC) codes

Computer Networks and Communications
Software

Access to Document

10.1145/3184558.3186921

Cite this

Chakraborty, T., Cui, Z., & Park, N. (2018). Metadata vs. Ground-truth: A Myth behind the Evolution of Community Detection Methods. In The Web Conference 2018 - Companion of the World Wide Web Conference, WWW 2018 (pp. 45-46). (The Web Conference 2018 - Companion of the World Wide Web Conference, WWW 2018). Association for Computing Machinery, Inc. https://doi.org/10.1145/3184558.3186921

@inproceedings{de4e2360af994bad9d32073a05dea0d5,

title = "Metadata vs. Ground-truth: A Myth behind the Evolution of Community Detection Methods",

abstract = "A community detection (CD) method is usually evaluated by what extent it is able to discover the 'ground-truth' community structure of a network. A certain 'node-centric metadata' is used to define the ground-truth partition. However, nodes in real networks often have multiple metadata types (e.g., occupation, location); each can potentially form a ground-truth partition. Our experiment with 10 CD methods on 5 datasets (having multiple metadata-based ground-truth partitions) show that the metadata-based evaluation is misleading because there is no single CD method that can outperform others by detecting all types of metadata-based partitions. We further show that the community structure obtained from the CD methods is usually topologically stronger than any metadata-based partitions. Finally, we suggest a new task-based evaluation framework for CD methods and show that a certain type of CD methods is useful for a certain type of task.",

author = "Tanmoy Chakraborty and Zhe Cui and Noseong Park",

note = "Funding Information: ACKNOWLEDGMENTS The work was partially supported by the Ramanujan Fellowship, SERB-DST, Govt. of India. REFERENCES Publisher Copyright: {\textcopyright} 2018 IW3C2 (International World Wide Web Conference Committee), published under Creative Commons CC BY 4.0 License.; 27th International World Wide Web, WWW 2018 ; Conference date: 23-04-2018 Through 27-04-2018",

year = "2018",

month = apr,

day = "23",

doi = "10.1145/3184558.3186921",

language = "English",

series = "The Web Conference 2018 - Companion of the World Wide Web Conference, WWW 2018",

publisher = "Association for Computing Machinery, Inc",

pages = "45--46",

booktitle = "The Web Conference 2018 - Companion of the World Wide Web Conference, WWW 2018",

}

Chakraborty, T, Cui, Z & Park, N 2018, Metadata vs. Ground-truth: A Myth behind the Evolution of Community Detection Methods. in The Web Conference 2018 - Companion of the World Wide Web Conference, WWW 2018. The Web Conference 2018 - Companion of the World Wide Web Conference, WWW 2018, Association for Computing Machinery, Inc, pp. 45-46, 27th International World Wide Web, WWW 2018, Lyon, France, 18/4/23. https://doi.org/10.1145/3184558.3186921

Metadata vs. Ground-truth: A Myth behind the Evolution of Community Detection Methods. / Chakraborty, Tanmoy; Cui, Zhe; Park, Noseong.
The Web Conference 2018 - Companion of the World Wide Web Conference, WWW 2018. Association for Computing Machinery, Inc, 2018. p. 45-46 (The Web Conference 2018 - Companion of the World Wide Web Conference, WWW 2018).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Metadata vs. Ground-truth

T2 - 27th International World Wide Web, WWW 2018

AU - Chakraborty, Tanmoy

AU - Cui, Zhe

AU - Park, Noseong

N1 - Funding Information: ACKNOWLEDGMENTS The work was partially supported by the Ramanujan Fellowship, SERB-DST, Govt. of India. REFERENCES Publisher Copyright: © 2018 IW3C2 (International World Wide Web Conference Committee), published under Creative Commons CC BY 4.0 License.

PY - 2018/4/23

Y1 - 2018/4/23

N2 - A community detection (CD) method is usually evaluated by what extent it is able to discover the 'ground-truth' community structure of a network. A certain 'node-centric metadata' is used to define the ground-truth partition. However, nodes in real networks often have multiple metadata types (e.g., occupation, location); each can potentially form a ground-truth partition. Our experiment with 10 CD methods on 5 datasets (having multiple metadata-based ground-truth partitions) show that the metadata-based evaluation is misleading because there is no single CD method that can outperform others by detecting all types of metadata-based partitions. We further show that the community structure obtained from the CD methods is usually topologically stronger than any metadata-based partitions. Finally, we suggest a new task-based evaluation framework for CD methods and show that a certain type of CD methods is useful for a certain type of task.

AB - A community detection (CD) method is usually evaluated by what extent it is able to discover the 'ground-truth' community structure of a network. A certain 'node-centric metadata' is used to define the ground-truth partition. However, nodes in real networks often have multiple metadata types (e.g., occupation, location); each can potentially form a ground-truth partition. Our experiment with 10 CD methods on 5 datasets (having multiple metadata-based ground-truth partitions) show that the metadata-based evaluation is misleading because there is no single CD method that can outperform others by detecting all types of metadata-based partitions. We further show that the community structure obtained from the CD methods is usually topologically stronger than any metadata-based partitions. Finally, we suggest a new task-based evaluation framework for CD methods and show that a certain type of CD methods is useful for a certain type of task.

UR - http://www.scopus.com/inward/record.url?scp=85052761200&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85052761200&partnerID=8YFLogxK

U2 - 10.1145/3184558.3186921

DO - 10.1145/3184558.3186921

M3 - Conference contribution

AN - SCOPUS:85052761200

T3 - The Web Conference 2018 - Companion of the World Wide Web Conference, WWW 2018

SP - 45

EP - 46

BT - The Web Conference 2018 - Companion of the World Wide Web Conference, WWW 2018

PB - Association for Computing Machinery, Inc

Y2 - 23 April 2018 through 27 April 2018

ER -

Metadata vs. Ground-truth: A Myth behind the Evolution of Community Detection Methods

Abstract

Publication series

Conference

Bibliographical note

All Science Journal Classification (ASJC) codes

Access to Document

Other files and links

Fingerprint

Cite this