IB Mathematics AI AHL Design of valid data collection methods MAI Study Notes - New Syllabus

IB Mathematics AI AHL Design of valid data collection methods MAI Study Notes

LEARNING OBJECTIVE

Design of valid data collection methods, such as surveys and questionnaires.

Key Concepts:

Selecting/Choosing relevant variables from many variables.
Categorizing numerical data
Definition of reliability and validity.

MAI HL and SL Notes – All topics

Purpose:
To gather structured, unbiased, and relevant information from participants.

Key Features of Good Design:

Unbiased phrasing (avoid leading questions)
Structured layout (clear sections and question flow)
Consistent answer choices (e.g., Likert scale)
Precision in questioning (specific and measurable)

Example:

Bad: “Don’t you think the AI tutor is great?”
Good: “On a scale of 1–5, how would you rate the effectiveness of the AI tutor in helping you understand mathematical concepts?”

Tip:
Pilot Test the survey on a small group to check clarity and identify issues.

Definition:

Choosing the most relevant variables (also called features) for your AI model or statistical analysis.

Goals:

Maximize relevance to the outcome
Eliminate irrelevant or redundant variables
Consider cost and feasibility of measurement

Example:
Predicting student performance in math:

Relevant: Previous grades, time on homework, attendance
Irrelevant: Hair color, favorite food, number of siblings

Importance:

Only appropriate, clean, and representative data should be analyzed to ensure valid outcomes.

Key Criteria:

Accuracy (no errors)
Completeness (no missing values)
Consistency (uniform format)
Relevance (aligned with research goals)
Representativeness (sample mirrors the target population)

Note:

In AI: “Garbage in, garbage out.” Low-quality data leads to unreliable models.

Used for:

Analyzing categorical data to test for independence or goodness of fit.

Good Practices:

Expected frequencies > 5 for each cell (otherwise results are unreliable)
Logical grouping of data (e.g., age ranges: 0–10, 11–20)
Avoid:
- Too few categories → loss of detail
- Too many → small expected frequencies → reduced statistical power

Definition:

Number of independent values that can vary in a calculation.

Goodness of Fit Test:

$\text{df} = \text{number of categories} – 1 – \text{number of parameters estimated}$

Test for Independence (contingency table):

$\text{df} = (\text{rows} – 1) \times (\text{columns} – 1)$

Common Mistake:

Forgetting to subtract estimated parameters → incorrect df → wrong conclusions.

Definition of Reliability

Reliability = Consistency

A reliable method yields consistent results under the same conditions.

Types of Reliability Tests (with Examples)

Type	Description	Example
Test–Retest	Same test, same group, different times	A university gives the same math test at the beginning and end of a semester (with no new teaching). High correlation = high test–retest reliability.
Parallel Forms	Two equivalent tests	Driving license exams with two question versions. Similar scores = strong reliability.
Inter-Rater	Multiple raters, same task	Gymnastics judges score routines independently. High agreement (e.g., Cohen’s Kappa) = strong reliability.
Internal Consistency	Similarity of items within one test	A 20-question anxiety test shows strong correlation between items (via Cronbach’s Alpha), indicating internal consistency.

Definition of Validity

Validity = Accuracy

How well a method measures what it is supposed to measure.

Types of Validity Tests (with Examples)

Type	Description	Example
Content Validity	Covers all aspects of the construct	A physics exam is reviewed by experts to ensure it tests all key topics like Newton’s laws and motion.
Criterion Validity	Correlation with external outcome	A company uses an aptitude test and compares scores with job performance after 6 months (predictive validity).
Construct Validity	Measures intended theoretical concept	A new depression scale correlates with brain scans and established tools → shows construct validity.
Face Validity	Appears to measure the right thing	A customer satisfaction survey asks about delivery, pricing, and service. People easily recognize its purpose.

Distinguishing Between Reliability and Validity

Concept	Meaning	Can it exist without the other?	Example
Reliability	Consistency	Yes – a test can be reliable but not valid	A miscalibrated scale always shows 5kg too heavy → reliable but invalid
Validity	Accuracy	No – a valid test must be reliable	A tutor gives very different scores each time → not reliable, so can’t be valid

Conclusion: