Make an e2e test for a standalone open source version to minimize "testing-in-production" adventures

Maybe I'm putting too much effort into it, but it "feels" better when my CI checks that I haven't done any obviously stupid things to #nullitics with my midnight commits. I promise I wouldn't spend more than 2 hours on this.