Heh… I’ve just completed NIH certification to administer human test protocols. I feel like that’s got to be worth a few mad scientist points.
So, I’m working on this project at work involving research into new forms of computer interface peripherals. (No, not that one.) As part of the project, we are required to test these interface concepts with live humans, survey them and see how well it works for them. The survey is positively harmless: basically like handing someone a shiny new cellphone set to vibrate, and programmed to vibrate in a different pattern depending who’s calling. Then run each pattern and see if the user can use it to successfully identify the caller. To make things simple, we had proposed to do these tests with our own employees (everyone pretty much said, “hell yeah, I’d like to play with that” when I described the project, and it sure as hell beats pulling in random strangers from the Craigslists to nose around in our secret lab).
Oh, for the catch: it’s a DARPA-funded project, and this required usability survey stuff constitutes Human Testing. To do any Human Testing, you need to draft a Human Test Protocol detailing exactly what you hope to get out of the research, intricate details of the exact test(s) that will be performed (down to the specific questions you will ask, and in what order) and the exact hardware involved in each test (note, this hardware hasn’t yet been designed by the time these forms/tests must be finalized…also remember that the entire point of these tests is to guide the design of said hardware, which now must be set in stone long before the test). So much for iterative development, that’s for sure. Add to that lengthy Informed Consent forms, confidentiality procedures, SOPs, HIPAA forms, risk assessments relative to daily activities (like, say, using your cell phone…), Conflict of Interest forms, Anti-Coercion Forms, and the whole shebang must be submitted to, and approved by, an IRB (Institutional Review Board), then the IRB’s approval approved by the governmental Office of Something Something Something. The package we submitted went up to Appendix V. (Not V as in roman numeral 5, V as in A thru… . I kid you not.). We each also had to take this lengthy “Protecting Human Research Participants” online course. Not that I’m not completely for protecting my human research participants, but come on – we’re asking them simple gadget usability questions, not giving them experimental drugs.
We got feedback from the IRB today, our protocol is NOT yet approved. They need more forms submitted and a bunch more specific details carved in stone.
I shouldn’t be complaining. Poor Jeff, my officemate, is handling the vast majority of this process in concert with a PhD consultant who is actually used to dealing with this crap. It just burns having to scramble around slapping together super-crude prototypes to test on-the-sneak to get even some rough semblance of how to structure the tests, not to mention the hardware, to get any kind of meaningful results (screwing the schedules on my other projects), how much of the project’s fixed budget will be swallowed up by this approval process instead of, say, actual R&D, and just how pointless the tests will be when it’s all said and done. Our money shot is 3 days’ worth of usability testing at the end of the program, predetermined at the start of the program.
Oh yeah, and letting your own staff participate instead of random losers from off the street? BIG MISTAKE. Far from simplifying the random Craigslist idiot, asking strangers medical screening questions, losers breaking our equipment, NDA, and losers selling our tech secrets to the Commies cans of worms, it actually hugely complicates things. Now you need to prove to the IRB, and the governmental IRB approval approvers, that employees’ participation in the research is indeed voluntary (not, “hey, participate in my research study or ur fired”), and all the steps you have taken to ensure that all participation is voluntary, that no one has the capability to bribe or coerce anyone into participating, that supervisors/bosses cannot determine who participated, etc.
Some notes, in case I EVER have to do this again:
Do Not promise, or get involved in a project where they will make you promise, human testing (no matter how harmless it seems) for any government project.
Do Not promise human testing -of anything- on a Phase I SBIR budget.
Do not write, review, modify or proofread human test protocols after playing Portal. Now the whole thing is in GLaDOS Voice. Likewise for the Big Red (supercolliding super-) Button that stops the test. (“The effects prolonged exposure to the e-stop are not a part of this test.”) I’m incredibly tempted to slip something into the protocol promising cake after the study, but then I’d probably have to get FDA approval too :-(
HIPAA stands for Highly Increased Paperwork and Annoyance Act.
(And yes, technically I *could* construct a test sequence in such a way as to make the prototype play music. But I haven’t gotten around to it yet choose not to. Yeah, that’s it.)
Leave a Reply