9.0 Evaluation🔗⧉

Contents of this part

• 9.1 Developing, testing and debugging
  • 9.1.1 The workflow: build, run, look at the picture
  • 9.1.2 Principles
  • 9.1.3 Test on inputs you can verify by hand
  • 9.1.4 Finding good real test inputs
  • 9.1.5 Per-algorithm debug recipes
  • 9.1.6 Crashes and bounds
  • 9.1.7 Vibe coding: writing image code with an LLM
• 9.2 The expert's eye as evaluation
  • 9.2.1 Judge it yourself first
  • 9.2.2 You are the expert, in a narrow sense
  • 9.2.3 Training your eye
  • 9.2.4 The input set decides what you can see
  • 9.2.5 Beware the average, look at the worst case
  • 9.2.6 Knowing when to stop, and when to ask
• 9.3 Image metrics
  • 9.3.1 Full-reference vs. no-reference
  • 9.3.2 Why not just L2?
  • 9.3.3 PSNR
  • 9.3.4 SSIM
  • 9.3.5 VDP and HDR-VDP
  • 9.3.6 Learned metrics
  • 9.3.7 The right metric depends on the task
• 9.4 Towards meaningful user studies
  • 9.4.1 How we got here
  • 9.4.2 Should you run a study at all?
  • 9.4.3 Three kinds of user research
  • 9.4.4 Comparing outputs, and the checklist that makes it mean something
  • 9.4.5 When the tool is the contribution
  • 9.4.6 Asking before building
  • 9.4.7 The rest of the toolbox
  • 9.4.8 Reporting, and calling it what it is
  • 9.4.9 Doing it ethically
  • 9.4.10 Why these studies fail to replicate

▸ full collapsible outline of this part