Felix's Robot Testing Discussion

A disucssion/introduction to basic testing.

Minutes

  • Testing is a state of mind
  • Unit test
    • Test should allow you to pinpoint where the problem is
    • If someone changes the implementation, it should come out correctly still
    • You can make these before you write the function (based off the spec)
  • Good tests find bugs
    • You probably don't want to toss a bunch of a printf's everywhere
  • Regression Testing
    • Write a test that fails reliably with the current code (that has a bug)
    • Collect these, everytime you modfify the code run these tests again
      • If they fail, you broke something that was originally fixed
      • Otherwise you should be good for now
    • Lots of suites to do this, but this may not work for Colony
  • Robot Example
    • Tell it to do something (drive forward x inches)
    • Check to see if it did it (did it make it to x inches within acceptable bounds?)
  • Think about what you want to test
  • The thought was to do a unit test for each code after you wrote it
    • This rarely works (because testing sucks)
    • Write the test and then write the code
    • Write the code and write the test immediately afterwards
    • If you find a bug, add more tests
    • If you change conventions, you might need to fix a test (the test may be wrong)
  • What if you don't know the right answer?
    • Gather data first, then have the human choose the best
    • Afterwards you can automate once you know what the right answer is
  • Validation testing
    • Use asserts to have sanity checks
      • Throw errors over serial if there is a problems
      • Continue running though, not exit
    • Might add a lot of noise to the code
    • Check inputs, outputs, states (internal/external), data is good
    • Can disable with flags
    • If you have too many you might kill the robot
    • Invalid asserts can be really bad...make sure they are legit
  • Document what you're expecting
  • Testing doesn't hurt
    • It doesn't need to be super vigorous
    • As long as your confident on the inputs it should be okay
    • Regression and unit tests are useful for later though
  • Automated builds may be hard for robots
  • You can fix things incrementally, don't go all at once
    • As things break, add regression tests