Table 1 Overview of current practices using open-ended items in ESM
Study element | Decision point | Representative options | Description | Example publication(s) |
|---|---|---|---|---|
Data collection | What information to capture? | General event descriptions | Participants recall the most important or emotionally salient event since the last assessment | |
Specific types of events | Participants only report on particular kinds of experiences | |||
Immediate context and experience | Participants describe what they are doing, where they are, who they are with, current thoughts and feelings | |||
Response processes | Participants explain how or why they responded as they did | |||
What format to use? | Fully open | Participants are prompted for freely-generated, running text | ||
Fully open (list) | Participants are prompted to provide short phrases or lists | |||
Partially open | Participants are provided with multiple-choice items, including an editable ‘other’ option | |||
How to integrate with other items? | Standalone | Responses can be interpreted independently of other items | ||
Follow-up | Responses expand upon responses to other, closed-ended items | |||
What type of data to collect? | Typed text | Participants write their responses using a computer or phone keyboard | ||
Voice recordings | Participants speak their responses into a phone or recording device | |||
Non-linguistic data | Participants upload additional media (e.g., images) | |||
Data analysis | How will data be processed? | Deductive manual coding | Researchers classify participant responses based on established frameworks or prior work | |
Inductive manual coding | Researchers derive response categories from the data | |||
Combination of deductive and inductive | Researchers add new response categories, based on the data, to a predefined list | |||
Automated coding | Responses are classified or scored using Natural Language Processing techniques, such as word counting programs and Large Language Models | |||
How will data be analyzed? | Descriptive statistics | Raw or coded responses are quantitatively characterized | ||
Inferential statistics | Coded responses are used to predict outcomes of interest or compare across conditions/groups | |||
Qualitative interpretation | Researcher observations are verbally described, guided by example quotes | |||
How will data be presented? | Illustrative quotes | Raw responses are selected to highlight key observations | ||
Visualizations | Distributions of coded responses are illustrated using pie charts, word clouds, Venn diagrams, etc. |
