CALL FOR RESOURCE PAPERS


We invite researchers to submit their resource papers that highlight the latest datasets, data models, and techniques that enable intelligent decision-making, predictive analytics, and machine learning.
Key Dates
All deadlines are at 11:59pm in the Anywhere on Earth timezone.
  • Abstract submission for Resource papers: 06 June 2023
  • Resource Papers Deadline: 16 Jun 2023
  • Papers Notifications: 1 Aug 2023
  • Camera Ready Deadline: 15 Aug 2023
Topics of Interest

We welcome submissions on all topics in the general areas of artificial intelligence, machine learning, data science, databases, information retrieval, and knowledge management. 

An ideal resource paper's topics of interest include, but are not limited to, the following areas:

  • Data resources comprising a new and innovative dataset or protocol, or one created using novel methods and/or algorithms
  • Data resources labeled using novel and well-described annotation and/or crowdsourcing approaches;
  • Software resources to support research on novel application domains or support novel evaluation tasks;
  • Software resources such as prototypes ad services, open source frameworks, or tools and libraries which support tasks and evaluation in data science, data engineering, or information & knowledge management.

Submission Guidelines

Resource papers must be no more than 4 pages long, plus unlimited references (note that additional appendices are not allowed). Citing supplementary materials that are accessible online via platforms like osf.io or GitHub is permissible. However,, reviewers have the discretion to decide whether or not they will review such materials. We cannot guarantee that these materials will be taken into consideration during the review process. The review of the resource papers will be single-blind, which means that the authors should include their names and affiliations in the paper. All submissions will be reviewed by the Program Committee of the Resource Track, who will evaluate the novelty of the technical features and/or research being presented, the research and/or development challenges, its expected impact, and its timeliness and relevance for the CIKM audience of practitioners and researchers. 

Manuscripts should be submitted to CIKM 2023’s Easychair site in PDF format using the ACM sigconf template, see https://www.acm.org/publications/proceedings-template. At least one author of each accepted paper must register to present the work as scheduled in the conference program. 

All papers should be submitted via Easychair: https://easychair.org/conferences/?conf=cikm23

Guidelines and review rubric for Resource papers

Resource papers reporting on a dataset must publish the datasets and metadata using a dataset-sharing service (e.g., Zenodo, Datorium, Dataverse, or any other dataset-sharing service that indexes your dataset and metadata and increase the re-findability of the data) that provides a DOI for the dataset, which should be included in the dataset paper submission. Ethical considerations must be discussed. Authors are encouraged to include a description of how they intend to make their datasets FAIR[1]. We would also encourage authors to consider addressing the questions covered in the Datasheets for Datasets recommendations[2]. 

The reviewing guidelines for the resource paper track will focus on the following criteria:

Novelty

  • What is new about this resource?
  • Does the resource represent an incremental advance or something more dramatic?

Availability

  • Is the resource available to the reviewer at the time of review?
  • Are there discrepancies between what is described and what is available?
  • Are the licensing/terms of use sufficiently open to allow most academic and industry researchers access to the resource?
  • If the resource is data collected from people, do appropriate human subjects control board procedures appear to have been followed?

Utility

  • Is the resource well documented? What level of expertise do you expect is required to make use of the resource?
  • Are there tutorials or examples? Do they resemble actual uses, or are they toy examples?
  • If the resource is data, are appropriate tools provided for loading that data?
  • If the resource is data, are the provenance (source, pre-processing, cleaning, aggregation) stages clearly documented?

Predicted Impact

  • What CIKM research activity is enabled by the availability of this resource?
  • Does the resource advance a well-established research area or a brand new one?
  • Do you expect that this resource will be useful for a long time, or will it need to be curated or updated? If the latter, is that planned?
  • How large is the (anticipated) research user community? Will that grow or shrink in the next few years?
Clarification on Large Language Model Policy

We follow the guidelines from the Program Chairs about the fair use of Large Language Models (LLMs), which can be found in the main call for papers. Papers that include text generated from a large-scale language model (LLM), such as ChatGPT, are prohibited unless the produced text is presented as a part of the paper’s experimental analysis. In essence, the policy details the following:

  • The Large Language Model (LLM) policy for CIKM 2023 prohibits text produced entirely by LLMs (i.e., “generated”). This does not prohibit authors from using LLMs for editing or polishing the author-written text. 

  • The LLM policy is largely predicated on the principle of being conservative with respect to guarding against potential issues of using LLMs, including plagiarism.

  • The LLM policy applies to CIKM 2023. We expect this policy to evolve in future conferences as we better understand LLMs and their impacts on scientific publishing.  
Ethics of Resource Type Papers

Resources are expected to be available as described, where “available” means that most researchers in our community could obtain and make use of the resource without strongly limiting the research they can perform with it. Datasets are expected to be collected in accordance with institutional review board standards and ACM standards of ethics. Reviewers are instructed not to use their reviews as an advocacy platform for these issues but to do what they can to help authors bring their resources to fruition.

Disclosure of Competing Interests

Disclosure of funding and competing interests: Authors are required to provide an explicit disclosure of funding (financial activities supporting the submitted work) and competing interests (related financial activities outside the submitted work) that could result in conflicts of interest, in a section (e.g., “Acknowledgments”) that should be added to the camera-ready version of accepted papers, but not in the version submitted for review (in order to maintain author anonymity). Furthermore, authors are required to read the AAAI code of conduct and ethics guidelines; submitting to CIKM implies that the authors agree to abide by these rules.

Dual Submission Policy

It is not allowed to submit papers that are identical (or substantially similar) to versions that have been previously published or accepted for publication, or that have been submitted in parallel to other conferences (or any venue with published proceedings). Such submissions violate our dual-submission policy. There are several exceptions to this rule:

  • Submissions are permitted for papers presented or to be presented at conferences or workshops without proceedings or with only abstracts published.

  • Submission is permitted for papers that have previously been made available as a technical report or similar, e.g., on arXiv. In this case, the authors should not cite the report so as to preserve anonymity.

ACM Policy Against Discrimination

All authors and participants must adhere to the ACM's discrimination policy. For full details, please visit this site: https://www.acm.org/special-interest-groups/volunteer-resources/officers-manual/policy-against-discrimination-and-harassment 

As a published ACM author, you and your co-authors are subject to all ACM Publications Policies https://www.acm.org/publications/policies, including ACM's new Publications Policy on Research Involving Human Participants and Subjects https://www.acm.org/publications/policies/research-involving-human-participants-and-subjects 
Organization and Logistics

All poster and demo presentations are planned as in-person events. In cases of unavoidable and unexpected travel restrictions (COVID related or otherwise), an exception may be made to allow authors to present their work remotely.
Resource Chair Contact Information

For more information, contact the appropriate chairs:

1. Wilkinson, M.D., Dumontier, M., Aalbersberg, I.J., Appleton, G., Axton, M., Baak, A., Blomberg, N., Boiten, J.W., da Silva Santos, L.B., Bourne, P.E. and Bouwman, J., 2016. The FAIR Guiding Principles for scientific data management and stewardship. Scientific data, 3(1), pp.1-9.

2. Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J.W., Wallach, H., Iii, H.D. and Crawford, K., 2021. Datasheets for datasets. Communications of the ACM, 64(12), pp.86-92.

Follow us on Twitter and Mastodon.