Introduction

This guide serves as a guide for researchers looking to utilise ChatGPT for the categorisation of open-text responses collected from surveys or questionnaires. With the integration of AI, it is now possible to efficiently process and categorise qualitative data, providing insightful analysis and a structured overview of respondent feedback. This guide outlines the steps necessary to generate and apply categories to open-text responses using ChatGPT, enabling researchers to incorporate these findings into broader data analysis efforts effectively.

Steps for Using ChatGPT for Categorising Open Text Responses

Generating Categories

1. Prepare Your Data: Compile a sample of 50-100 open-text responses to a specific survey question. This number ensures the quality of categorisation.
2. Initial Prompt to ChatGPT: Present the sample responses to ChatGPT along with the survey question. Ask for a list of 6-12 categories that are relevant to the question, adhering to the guidelines provided for specificity, sentiment indication, and mutual exclusivity.
3. Refine Categories: Review the suggested categories for any necessary refinements or adjustments to better match the themes within the responses.

Categorising Answers

1. Prepare the Prompt: Once categories are finalised, prepare a new prompt for ChatGPT that includes the categories and the same set of responses.
2. Inclusion of Categories in Prompt: Always include the list of categories within the prompt to ensure ChatGPT does not stray from the established context.
3. Coding Responses into Categories: Ask ChatGPT to code each response into the relevant category or categories. This should be done in table format, with the first column displaying the response and subsequent columns displaying the categories.
4. Quality Check: Perform a quality check on the categorised responses to ensure accuracy and completeness. Adjustments may be necessary if responses have been miscategorised.
5. Integration into Data Analysis: The final categorised data can then be copied and pasted into an Excel or Google Sheets document for further analysis or integration into larger data sets.

Notes

• Quality Preservation: Limiting responses to 50 at a time helps maintain the quality of categorisation. If more responses need processing, repeat the process in batches.
• Context Preservation: Including the categories in the prompt for the coding phase prevents ChatGPT from generating unintended categories and ensures consistency.
• Continuous Improvement: The categorisation framework may require ongoing adjustments as more data is processed. This iterative process ensures that the categorisation remains relevant and accurate over time.
• UK Spelling: Ensure all categories and prompts use UK English spelling to maintain consistency across the dataset.

The Prompts

Generating categories prompt

Below is a list of answers to an open text survey question. Respond with 6-12 categories, depending on how many are needed to code answers into. 

Guidelines:

Don't create a category for test data

• Specificity: Refine broad categories, e.g., “Financial Stability” into "Income Stability", "Expense Management", and "Financial Planning".
• Singular Terminology: Use singular terms, e.g., “Girl” instead of “Girls”.
• Sentiment Indication: Indicate sentiment, e.g., “Stable Family” and “Unstable Family” instead of “Family Stability”.
• Open-ended Categorization:Don't limit the number of categories. Create as many as necessary to cover the themes in the responses.
• Mutual Exclusivity: Ensure categories are mutually exclusive, e.g., “Academic Achievement” and “Career Advancement” instead of a broad “Success” category.
• Multi-categorization: Code responses into multiple categories if they touch on several themes.
• Consistency: Keep a consistent approach in categorization, with clear definitions for each category.
• Custom Categories: Create custom categories for unique or uncommon themes.
• Continuous Improvement: Revisit and refine the categorization framework as needed.
• UK Spelling: Return categories in UK spelling

-- ANSWERS BELOW --

‍

Example Run:

‍

Categorise answer prompt:

Given the following answers (below), identify the relevant categories they belong to by coding them into the relevant categories (displayed below). 

Give back in a table format with the response in the first column and the category in the 2nd, 3rd, 4th etc. 

If the response fits into more than 1 category, display the category word in the category column for the given row.

Answers:

{{insert}}

Categories:

{{insert}}

‍

Example Run:

‍

‍