Lesson 8: Build, Test, Improve: Model Rescue and Certification

Step 8.1 · Start here

In the first seven lessons, you learned how to guide AI, check its answers, look for bias, protect private information, and use AI responsibly. In Lesson 8, you will build, test, and improve a simple AI model. Then you will complete the final assessment.

In this lesson, you will choose a program area that interests you and work with one of six made-up workplace examples. You will build, test, and improve a simple AI model. You will find errors, suggest safety steps, and complete the final assessment.

💡 Big Question and Core Rule

Big Question: How can people train, test, improve, and oversee an AI ? How can they help it make accurate and fair choices at work?

Main Rule: Better data can improve a model, but people must remain in control. Do not use or trust a model until people test it, study its errors, and take responsibility for its use.

🧠 Memory Phrase: Quality data in, human oversight throughout.

🛡️ Use the Responsible AI Routine

Use this pathway whenever AI may support a task.

When to use this routine

Use it before AI supports a school, workplace, or personal task, especially when information, safety, fairness, or other people may be affected.

Your next move

Study the five steps in the image. Choose one step you think is easiest to forget and explain why it matters.

View accessible web version Download print version

Goal → Protect → Use → Check → Own

Goal: What workplace classification problem or task is the model trying to solve?
Protect: How do we protect system integrity, prevent harmful biases, and maintain data safety?
Use: Is the model trained on representative data, tested across diverse inputs, and used for its intended purpose?
Check: Have false positives, false negatives, and confidence scores been audited for accuracy and fairness?
Own: Who has final responsibility for approving the model, watching for errors, and making final decisions?

🎯 Learning Goals

By the end of this lesson, students can:

Choose a workplace example linked to your program. Then use to build a simple AI .
Test the model, calculate how often it is correct, and study its mistakes, including false positives and false negatives.
Improve the model by training it again with better examples that target its weak areas.
Suggest safety checks and clear rules for AI classifiers that are in use.
Show what you learned about responsible AI by completing the final assessment.

✅ You will show success by: selecting a program area, training and iteratively improving a , diagnosing model failure points, recommending human oversight procedures, and passing the final certification exam.

⏱️ Pacing at a Glance

Lesson 8 required activities and estimated time
Lesson Step	Time
8.2 Connect & Learn: How Machine Learning Models Classify Data	8 minutes
8.3 Learn: Model Rescue & Error Analysis (Build, Test, Improve)	12 minutes
8.4 Apply Your Learning: Model Rescue Performance Challenge	15 minutes
8.5 Show What You Know: Cumulative AI Literacy & Ethics Certification Exam	15 minutes
Total	50 minutes

📚 Key Words to Know

Model Training & Performance

: An AI system trained to sort inputs into specific categories or labels based on patterns in data.

: The initial set of labeled examples provided to an AI model so it can learn to recognize patterns.

: A probability or percentage metric generated by an AI model indicating how certain it is about a specific prediction.

Error Analysis & Governance

: Errors where a model incorrectly flags something as positive when it is not (false positive) or fails to detect a target condition (false negative).

: Improving a model by analyzing failure cases, adding targeted new data, and testing again.

: A system design where humans actively review, verify, or override automated AI outputs before final action is taken.

Source Foundation

Step 8.2 · Learn and check

Connect & Learn: When a Model Works Unevenly

Connect & Learn: When a Model Works Unevenly (5 minutes)

Estimated time: 5 minutes

Responsible AI Routine: Goal → Protect → Use → Check → Own

Scenario

A classifier gets four of six test records correct. Both mistakes happen under the same type of condition. A team member says, “Four out of six is good enough. We should use it.”

Reflect to yourself:

Is the model completely unsuccessful?
What does the overall result tell the team?
What does the repeated error pattern show that the total score hides?
What change to the could address this weakness? Who could be affected if the model is used too soon?

Key Idea

A total score can hide repeated failures under certain conditions or for certain groups. A responsible review checks both the total result and where the mistakes happen.

Last Updated: August 6, 2026

Fictional practice data showing 90 percent overall model accuracy while Test Group A scores 96 percent, Test Group B scores 91 percent, and Test Group C scores 68 percent. Review questions ask whether each group is represented, which errors matter most, and what must improve before use. — Compare the group results with the overall score. Notice what the 90 percent summary hides and decide what must be reviewed before use. Select the image to enlarge it.

✅ Your next move: Complete the checks for understanding below. Use the feedback to review your answer before continuing.

Check 1

An essay AI often gives low scores to essays about sports. What should the team do before deciding why?

Add sports essays because missing examples must be the cause. Stop using the tool because one pattern proves it cannot improve. Record the pattern, study causes, make one change, and test it. Remove sports essays so the scoring gap is no longer visible.

Check 2

A hospital AI predicts recovery time and is 90% accurate overall. Why should doctors study the errors? Choose all correct answers.

The average may hide repeated errors for one group. The overall rate explains the cause of every error. The tool may work worse for patients with rare conditions. One error proves unfair treatment across all groups.

Check 3

A classifier repeatedly misses examples photographed in low light. Which response is most responsible?

Approve it because its overall score is still high. Add appropriate low-light examples, retrain, and test again. Remove low-light tests so the reported score improves. Assume low light is the only possible cause and skip investigation.

Step 8.3 · Learn and check

Learn: The Six-Step Model-Building Cycle

Estimated time: 5 minutes

Responsible AI Routine: Goal → Protect → Use → Check → Own

Building an AI model is a cycle that repeats. People define the goal, prepare data, train and test the model, improve it, and monitor how it is used.

Six-step model-building cycle: 1 Define the goal and success criteria; 2 Collect and prepare appropriate data; 3 Train the model; 4 Test results and compare groups; 5 Improve data, rules, or training; 6 Monitor use and keep human responsibility. The cycle returns to step 1, and people make choices at every step. — Trace the cycle from Define through Monitor. Notice that human choices and responsibility continue at every step. Select the image to enlarge it.

Graphic formats: Open mobile version · Open print version

Define: Set the goal and success criteria.
Collect: Gather and prepare appropriate data.
Train: Build the model from the prepared examples.
Test: Check results and compare performance across groups or conditions.
Improve: Change the data, rules, or training when evidence shows a problem.
Monitor: Review real use and keep people responsible for decisions.

As you answer the questions below, think about how each step helps people find problems and improve the system.

✅ Your next move: Complete the checks for understanding below. Use the feedback to review your answer before continuing.

Check 1

A team divides its bird photos into three batches. Match each batch with its role in developing and evaluating the model.

Training data

Validation data

Test data

Check 2

A factory's AI robot repeatedly drops fragile glass items. Put the improvement process in the most responsible order.

Choose the steps in order, from first to last.

Step 8.4 · Learn and check

Learn: Compare Model Types and Check Tool Fit

Estimated time: 5 minutes

Responsible AI Routine: Goal → Protect → Use → Check → Own

Not all AI tools do the same job. When you face a task in your field, choosing the right tool, or deciding not to use AI at all, helps you work safely and efficiently. Review the model types below before answering the reflection questions.

Model-Type Reference

Table comparing classifier, predictor, recommender, generative, and rule-based tool types, including their ideal applications and potential drawbacks.
Model / Tool Type	Best Used For	Main Limitation
Classifier	Sorting an item into a set group or category.	May place tricky or unusual items into the wrong group.
Predictor	Estimating a future result or unknown number.	Guesses can be wrong when real-world conditions change.
Recommender	Ranking options or suggesting choices based on data.	May repeat narrow choices or old habits over time.
Generative Model	Creating new text, images, code, or ideas.	Can sound confident but still give wrong, unsafe, or copied details.
Rule-Based / Non-AI	Using simple rules, barcodes, checklists, or human checks.	Does not learn on its own, but offers lower risk and higher accuracy for simple tasks.

The Tool-Fit Question

Which approach best fits your task: a classifier, predictor, recommender, generative model, rule-based tool, or non-AI process?

To decide, ask: What result do I need? How accurate must it be? What could happen if the tool is wrong? Also check privacy, energy use, and whether a simpler method can do the job safely.

Reflect on the reference guide above and answer the questions below to test your understanding.

Last Updated: August 6, 2026

✅ Your next move: Complete the checks for understanding below. Use the feedback to review your answer before continuing.

Check 1

A city is planning several AI-supported services. Categorize each required output by the model type that best fits the task.

Each road report receives one label: routine, urgent, or emergency

Each recycling image receives an accepted or rejected label

Transit planners receive a projected passenger count for next month

Maintenance staff receive an estimate of when a pump may fail

A resident receives a prioritized list of library programs based on stated interests

An inspector receives a ranked list of locations to examine first

Communications staff receive a new first draft of a public newsletter

A designer receives several newly created poster concepts from written requirements

Check 2

A store uses cheap, accurate barcodes. A manager wants an expensive AI camera instead. What should the manager do first?

Keep barcodes unless AI adds value for its cost and risk. Replace barcodes because AI can give more sales details. Use AI because newer tools are always more accurate. Remove labels so the camera is the only tracking tool.

Step 8.5 · Practical application

Apply Your Learning: Model Rescue Challenge

Responsible AI Routine: 1 Goal—What am I trying to do? 2 Protect—What should stay private? 3 Use—How can AI help? 4 Check—Is it accurate and safe? 5 Own—Who is responsible in the end? — Use the routine to guide your final model decisions and reflection. Select the image to enlarge it. Open web/mobile version Open print-quality version

Estimated time: 25–30 minutes | Grading: Complete / Incomplete

Purpose

In this challenge, you will train, test, and improve a simple AI classifier inside a web simulation. Record your results as you work. Then explain what changed and why a person must make the final decision.

What you will do: Run an Initial Model, add three better examples, run the Improved Model, and compare the results.

What you need: The Model Rescue simulation and the response worksheet.

What you will submit: Your completed worksheet with both results and four reflection answers.

Start here: Make a copy of the worksheet, then choose your program group below. Keep the worksheet open while you run the simulation so you can record both results.

📥 Make a Copy of the Worksheet

Safety & Privacy Rules

Use only the fictional data provided inside the challenge.
Never type real student, patient, customer, or employee names into the tool.
Treat all AI results as clues to double-check—never as final proof.

1. Choose Your Program Group

Choose the program group that best matches your current program or career interest. You make this choice. In the simulation, select the same group number shown in the table.

Program and Challenge Topics
Program Group	Career Programs	Challenge Topic
Program Group 1	Health Sciences, AgriScience, Criminal Justice	Case and Evidence Review
Program Group 2	Automotive, Mechatronics	Equipment Inspection
Program Group 3	Construction Trades, Welding	Worksite Readiness
Program Group 4	Computer Science, Graphic Design, Marketing	Digital Project Review
Program Group 5	Education, Public Service	Support and Response
Program Group 6	Cosmetology, Culinary, Hospitality	Client and Service Safety

2. Simulation Instructions

Step 1: Open & Train Initial Model
Launch the simulation, select the program group you chose from the table, and review its initial . Run the Initial Model and notice where it makes mistakes.
Step 2: Improve & Retrain (Improved Model)
Add three new data points from the Improvement Bank to fix the missing details or bias. Retrain the Improved Model and check your new accuracy score.

Launch the Web Simulation

Open the simulation in a new tab. Use your chosen program group. Record your Initial Model result before adding three examples. Then record your Improved Model result and return here.

Open Model Rescue Challenge (opens in a new tab)

3. Reflection & Submission

In your worksheet, first record your program group, Initial Model result, three improvement choices, and Improved Model result. Then answer the following four questions. Submit the completed worksheet using your instructor's directions:

Program Choice and Pattern: Which program group did you choose? What repeated mistake or missing pattern did you observe in the Initial Model?
Data Improvement: Compare your two results. How did the three examples you added change the Improved Model's performance?
Connecting Prior Knowledge: How does this connect to real-world work in your career area? Give one example of a bad decision that could happen if someone blindly trusted an uncorrected AI model in your field.
Human Responsibility: According to the Responsible AI Routine (Goal → Protect → Use → Check → Own), who holds ultimate responsibility for a decision made with the help of an AI tool, and why?

Step 8.6 · Lesson assessment

Show What You Know: AI Skills Certification Assessment

✅ Your next move: Return to your course and complete the Lesson 8 assessment. Follow your instructor's directions for attempts and the required score.

Lesson 8: Build, Test, Improve: Model Rescue and Certification

💡 Big Question and Core Rule

🛡️ Use the Responsible AI Routine

When to use this routine

Your next move

🎯 Learning Goals

⏱️ Pacing at a Glance

📚 Key Words to Know

Connect & Learn: When a Model Works Unevenly

Scenario

Key Idea

Check 1

Check 2

Check 3

Learn: The Six-Step Model-Building Cycle

Check 1

Check 2

Learn: Compare Model Types and Check Tool Fit

Model-Type Reference

The Tool-Fit Question

Check 1

Check 2

Apply Your Learning: Model Rescue Challenge

Purpose

⚠️ Safety & Privacy Rules

1. Choose Your Program Group

2. Simulation Instructions

🚀 Launch the Web Simulation

3. Reflection & Submission

Show What You Know: AI Skills Certification Assessment

Safety & Privacy Rules

Launch the Web Simulation