Christopher Allan Webber

Christopher Allan Webber at

the question is: what's more work for you, chris? your time is quite valuable!

Supporting sqlite has caused more headaches and slowdowns in MediaGoblin's design than anything else. Several months have gone into looking into building a half-our-own migration system, namely because sqlite is so bad at migrations that when we switched from Mongo->SQL, Sqlalchemy-Migrate was nearly dead and Alembic was the clear future of migration tooling in the sqlalchemy world, but we went with a hacked up sqlalchemy-migrate solution with a lot of custom code from me just so we could move forward. Later, Natalie Foust-Pilcher had to write some more custom solutions because certain kinds of migrations were actually not possible because sqlite would explode dramatically. As such, whether you're running postgres or sqlite, we have some hacky code that makes the migration system very slow and require O(n) time migrations where n is number of rows slow for each individual migration. Given our limited resources, supporting multiple clever paths is a bit too hard, so that sucks.

This has also lead to migrations in MediaGoblin being much more precarious and something I fear a lot more than they'd have to be.

This is the also most challenging thing holding up Python 3 support in MediaGoblin. The student who is handling the Python 3 port is having to handle moving us over and investigating how to handle sqlite stuff in Alembic. That wouldn't be a worry if we weren't dealing with it.

ON THE OTHER HAND: sqlite has made writing unit tests a breeze, has made setting up test instances trivial, and many other things. I really love sqlite! If it weren't for the alter table thing, it'd be just stellar.

My blogpost on the subject might have seemed a bit over the top. If you're responsible for dealing with the situation though, it isn't.

Also, regarding the "I don't want to have 20 different databases on my machine running!" I hear ya, that's a really bad situation. My current feeling is actually now that it looks like postgres is going to probably solve 98% of the current "NOSQL" document database cases, maybe the free software network services world should just decide that postgres is the de-facto database. That'd solve a lot of things.

Dan Scott, Jason Self, Andrew E likes this.