RL Robotics LabSearch ↗
Lesson / authored

Week 24 · Improve: Reflection chooses the next experiment

Session 4 of 4 · Demonstrate and defend · Phase 6

Plan about 15 minutes for explanation, 30 minutes for practical work and 10–15 minutes for documentation. A longer build may continue into the next session: stop safely, commit the current state and record the next check. Desktop simulations count as software evidence; label them clearly and record physical validation separately.

Engineering challenge

Can someone else understand, reproduce and challenge your engineering conclusions? This session focuses on reflection chooses the next experiment.

Before you start

The previous week’s recorded baseline and Week 23, Improve. For later sessions this week, retain the preceding session’s files and predictions.

Equipment: Desktop Python, editor, paper and ruler; for physical work, the configured 3pi+ 2040, clear floor mat and hardware checklist. Week 3 additionally uses the separate low-voltage LED circuit described in its procedure.

For any motion, verify the stop button, short time limit and clear floor area first. Keep the wheels raised for a new device program until its commands and stop behaviour are checked. A hazard or uncertain input is a reason to stop and document, not to force the trial to finish.

Theory and mathematics

Reflection chooses the next experiment

Reflection identifies what changed in your reasoning, which assumptions failed and which habit most improved reliability. Rank future work by the evidence gap it would close, not by novelty. A new sensor is useful only if its measurement changes a decision the robot currently cannot make. Preserve the final baseline and report before beginning an extension. The capstone can conclude with a limited or partially unsuccessful physical mission if the evidence and analysis are complete; do not rewrite failures into successes.

Worked example — illustrative values

A defensible claim is: “Version v1.0 arrived in 8 of 10 held-out trials on our surveyed mat; arrived endpoint error averaged 6 cm, with two hazard stops.” It names version, denominator, conditions and a conditional metric. “The robot always navigates accurately” is unsupported by that evidence. These figures are fictional examples, not course results.

Write the calculation in your notebook before running code. State which values you measured, which you assumed and which the program calculates. A correct numerical calculation cannot rescue an incorrect physical assumption.

Run and explain the model

The following is desktop Python, not a ready-to-run motor program. Download this week’s example, save it in your student repository and run python3 code/w24.py from the repository root. The same small model is reused across the week so you can learn it, build with it, test it and revise it.

# Desktop Python teaching example. Numerical inputs are illustrative.
def claim_summary(version, attempts, arrivals, conditions):
    if attempts <= 0 or not 0 <= arrivals <= attempts:
        raise ValueError("invalid trial counts")
    return (f"{version}: {arrivals}/{attempts} arrivals under {conditions}. "
            "See raw trials for errors, stops and limitations.")

print(claim_summary("example-v1", 10, 8, "fictional surveyed-mat tests"))

Compare the unchanged baseline and your proposed change on the same inputs. Keep the original files so another reader can reproduce the comparison. If an exception appears, read its final line, identify the input or assumption that caused it and make the smallest explained correction. Do not delete validation merely to obtain output.

Understanding the model and its limits

The summary function checks that counts are logically possible before forming a claim. It deliberately leaves detailed errors and stop reasons in the linked raw record, because a one-line summary cannot establish every result. Replace its fictional version and conditions with the exact archived build and evaluated envelope. Then audit the prose: can each number be reconstructed, and are failed attempts represented? A peer should locate the dataset and repeat a calculation without asking you for an undocumented file. An honest “not tested” is stronger than a confident statement unsupported by evidence.

Practical instructions

  1. Write a reflection on three changed assumptions and three effective engineering habits.
  2. Rank the next three improvements by evidence gap, effort and likely benefit.
  3. Archive a final tag, report, raw data, plots and known limitations.
  4. Complete the course checklist and identify one extension without altering the archived baseline.

Experiment

Compare your Week 1 reasoning with the final report; select one future experiment with a measurable decision criterion.

Before testing, record your prediction, changed factor, measured response, fixed conditions and stopping rule. Save every attempted run, including failures, with a condition and source version. If hardware is unavailable, use an explicitly labelled synthetic/replay dataset and list the physical question it cannot answer. Do not invent completed trials.

Deliverable

An archived final baseline, full report and a specific next experiment; no unsupported success claim.

Save notebook/w24-s4.md, the relevant code revision, raw CSV or test-case records, and one labelled diagram/plot/table. Link the files relatively from your notebook. Use the entry template and report guide.

Completion criteria

A documented failed prediction can meet the learning criteria. A missing physical trial must remain marked untested; software success alone does not validate the robot.

Reading and video

Reflection and next step

Which assumption most affected your result? Point to one observation that supports your explanation and one alternative explanation the evidence has not ruled out. Write a specific next test with a changed factor and measurable outcome, then proceed through the week’s Learn → Build → Experiment → Improve cycle.

← Previous sessionCourse roadmap