Don't always enable vm-memory/backend-mmap

[vireshk]

We enabled this recently, #175, to make the builds pass but maybe we should get around this with an alternative solution which doesn't require the forceful enabling here.