Testing and Validating Models

EES 4760/5760

Agent-Based & Individual-Based Computational Modeling

Jonathan Gilligan

Class #7: Tuesday January 30 2018

Organization

Organization

  • Team Projects:
    • Class Presentations Feb. 27
    • Choice of two projects:
      • Adaptive behavior (Business investor model, Ch. 10)
      • Agent interactions (Telemarketer model, Ch. 13)
    • Decide which one you want to do and choose a partner
      (undergrads with undergrads, grads with grads)

Finding and Fixing Errors

Classes of Errors

  • Typographical (typing pxcor when you mean pycor)

  • Syntax

  • Misunderstanding NetLogo language:

    ask turtle 5 [
      let neighbor-turtles turtles in-radius 2
      ask neighbor-turtles [set color green]
    ]

    versus

    ask turtle 5 [
      let neighbor-turtles turtles-on patches in-radius 2
      ask neighbor-turtles [set color green]
    ]
  • Wrong display settings (wrapping)

  • Run-time errors (e.g., division by zero, moving turtle out of world
    forgetting to initialize globals, turtles-own, or patches-own)

  • Logic errors (hard to find)

  • Formulation errors (hard to find)

Finding Errors

  • Syntax checks
  • Visual tests
  • Print statements (highly recommended)
    • print, show, type, write, …
    • output-print, output-show, … outputs to model output area instead of Command Center
    • file-print, file-show, … outputs to a file.
      • Must call file-open first
  • Spot tests with monitors on interface
  • Unit tests with jg-tif.nls

Chasing Down Errors

  • Stress tests
    • Run with many different extreme values using unit tests
  • Code reviews (teamwork)
  • Statistical analysis of output
    • BehaviorSpace
    • file-open, file-print, …
    • export-plot

Independent Re-Implementation of Submodels

  • If your model needs a tricky calculation:
    • Try it in another format: spreadsheet, scripting language (Python, R, Matlab, etc.)
    • Compare to NetLogo results

Marriage Model

Marriage Model ODD

  • Purpose: Describes social norms of age at which people marry
  • Entities, State-Variables, Scales: Agents are individual people.
    • Age (0–60), sex, marital status
    • Social angle: describes location in social network
      • Social network like cylinder: coordinates are age and social angle

social network: age goes from 0–65, social angle goes from 0–360