How can I pass JVM system properties on to my tests?

I have the following task

task testGeb(type:Test) {
   jvmArgs '-Dgeb.driver=firefox'
   include "geb/**/*.class"
   testReportDir = new File(testReportDir, "gebtests")
}

The system property doesn't appear to make it to the Geb tests, as Geb does not spawn Firefox to run the tests. When I set the same system property in Eclipse and run the tests, everything works fine.