RL Robotics LabSearch ↗
Lesson / authored

Week 16 · Experiment: Robustness is a matrix of conditions

Session 3 of 4 · Combine evidence · Phase 4

Plan about 15 minutes for explanation, 30 minutes for practical work and 10–15 minutes for documentation. A longer build may continue into the next session: stop safely, commit the current state and record the next check. Desktop simulations count as software evidence; label them clearly and record physical validation separately.

Engineering challenge

How should the robot act when different sensors give incomplete or conflicting evidence? This session focuses on robustness is a matrix of conditions.

Before you start

The previous week’s recorded baseline and Week 15, Improve. For later sessions this week, retain the preceding session’s files and predictions.

Equipment: Desktop Python, editor, paper and ruler; for physical work, the configured 3pi+ 2040, clear floor mat and hardware checklist. Week 3 additionally uses the separate low-voltage LED circuit described in its procedure.

For any motion, verify the stop button, short time limit and clear floor area first. Keep the wheels raised for a new device program until its commands and stop behaviour are checked. A hazard or uncertain input is a reason to stop and document, not to force the trial to finish.

Theory and mathematics

Robustness is a matrix of conditions

Reliability across conditions requires a test matrix: obstacle side, surface/lighting, approach angle and sensor availability. A single demonstration covers only one point in that matrix. Replaying recorded sensor streams lets you test policy changes without placing the robot in danger. Physical trials then check the complete pipeline, including sample timing, state transitions and actual stopping. Do not make two simultaneous changes to threshold and recovery while comparing policies. Keep configurations and raw data linked.

Worked example — illustrative values

If contact is true OR boundary is true, stop intent is true. If either measurement is stale, the policy also stops. With 18 successful approaches out of 20, observed success is 90%; it does not establish a 90% guarantee for all future routes.

Write the calculation in your notebook before running code. State which values you measured, which you assumed and which the program calculates. A correct numerical calculation cannot rescue an incorrect physical assumption.

Run and explain the model

The following is desktop Python, not a ready-to-run motor program. Download this week’s example, save it in your student repository and run python3 code/w16.py from the repository root. The same small model is reused across the week so you can learn it, build with it, test it and revise it.

# Desktop Python teaching example. Numerical inputs are illustrative.
def should_stop(contact, boundary, fresh):
    return not fresh or contact or boundary

for contact, boundary, fresh in [(False, False, True), (True, False, True),
                                  (False, True, True), (False, False, False)]:
    print(contact, boundary, fresh, "stop", should_stop(contact, boundary, fresh))

Use your own recorded data or named test fixtures instead of the illustrative inputs. Save expected and actual values side by side. If an exception appears, read its final line, identify the input or assumption that caused it and make the smallest explained correction. Do not delete validation merely to obtain output.

Understanding the model and its limits

The Boolean stop rule is intentionally conservative: contact, a floor boundary or nonfresh evidence each blocks motion. Only all-clear fresh evidence permits continued intent. Enumerating all eight combinations proves this small logical contract, but not the physical detection quality. A sensor can supply a fresh but wrong reading. Keep calibration/confusion tests alongside the truth table. Do not average unrelated quantities such as encoder counts and reflectance values into one confidence number. Combine their physical meanings: pose supports route progress, floor sensing supports boundary guards and contact sensing supports an immediate stop.

Practical instructions

  1. Build a matrix with four conditions, such as clear floor, tape boundary, left contact and right contact.
  2. Run five sensor replays or safe low-speed fixture approaches per condition.
  3. Keep policy, calibration and speed fixed; label each actual condition independently.
  4. Classify successful detections, false stops, missed hazards and safe aborts.

Experiment

Test 20 labelled approaches/replays in four groups; compare policy outcomes without changing settings mid-test.

Before testing, record your prediction, changed factor, measured response, fixed conditions and stopping rule. Save every attempted run, including failures, with a condition and source version. If hardware is unavailable, use an explicitly labelled synthetic/replay dataset and list the physical question it cannot answer. Do not invent completed trials.

Deliverable

20 classified outcomes with separate false-stop and missed-hazard reporting.

Save notebook/w16-s3.md, the relevant code revision, raw CSV or test-case records, and one labelled diagram/plot/table. Link the files relatively from your notebook. Use the entry template and report guide.

Completion criteria

A documented failed prediction can meet the learning criteria. A missing physical trial must remain marked untested; software success alone does not validate the robot.

Reading and video

Reflection and next step

Which assumption most affected your result? Point to one observation that supports your explanation and one alternative explanation the evidence has not ruled out. Write a specific next test with a changed factor and measurable outcome, then proceed through the week’s Learn → Build → Experiment → Improve cycle.

← Previous sessionCourse roadmapNext session →