Integration Testing Entity Framework code first with in-memory database

I'd like to run actual integration tests of my EF4.1 repositories against an in-memory database a la ayende's nhibernate version.

I have a code first model, against a legacy database (old table and column names need mapping to my entites using code configurations).

I'd like to be able to use Sqlite (or other) to:

1. Generate an in-memory database from my model
2. Create a DBContext for my model with this in-memory database
3. I have already in place IoC/DI of a IDBContextFactory which gets constructed with my (Generic) Repositories (also using a GenericRepository pattern)

There's bits and bobs on-line which suggest it should be possible, but not much for code-first approaches. Anyone know if this is possible?

Some snippets of my test library, see // THROWS ERROR marking runtime errors:

public class MyDbContextFactory : IDbContextFactory
  {
    private static object context;
    public object CurrentContext
    {
      get {
        if(context == null)
        {
          // ?? DOESN'T WORK AS THERE'S NO META DATA
          var connBuilder = new EntityConnectionStringBuilder();
          connBuilder.Provider = "System.Data.SQLite";
          connBuilder.Metadata = 
           @"res://*/TestEfDb.csdl|res://*/TestEfDb.ssdl|res://*/TestEfDb.msl";
          connBuilder.ProviderConnectionString = 
           ConfigurationManager.ConnectionStrings["DataContext"].Name;

          var entConnection = new EntityConnection(connBuilder.ConnectionString);

          // THROWS ERROR: sqlite Format of the initialization string does not
          // conform to specification starting at index 0
          // for connection string "Data Source=:memory:;Version=3;New=True;"

          //var entConnection = new EntityConnection
          // (ConfigurationManager.ConnectionStrings["DataContext"].Name);
          context = new MyDbContext(entConnection);
        }
        return context;
      }
    }
  }

...

 [Test]
    public void test_me()
    {
        var auditRespository = new AuditRepository(new MyDbContextFactory());
        auditRespository.GetAll();
    }