Ubuntu Wiki

   1 [18:01] <aquarius> first, thanks to the Alexes :-)
   2 [18:01] <aquarius> The subtitle for this talk on the schedule is "synced data for your apps on many platforms", and that's what u1db does.
   3 [18:01] <aquarius> You build an app, and use u1db to store its data, and you can then sync that data to your app on other machines and other devices and other platforms.
   4 [18:02] <aquarius> So if you built an Ubuntu app, you could have the same data sync between all your Ubuntu machines, meaning that your data is everywhere.
   5 [18:02] <aquarius> You could also sync to other platforms, so an Android app could share the same data, and so on.
   6 [18:02] <aquarius> Imagine your shopping lists and your notes and your movie watching habits and your music ratings in your Ubuntu desktop and your smartphone and on the web available from anywhere.
   7 [18:03] <aquarius> So far, this is just syncing, like Ubuntu One file sync: why have a database?
   8 [18:03] <aquarius> U1DB is good at dealing with data; file sync isn't.
   9 [18:03] <aquarius> Some people have thought in the past "hey, my app stores things in SQLite; I'll just use U1 to sync my app's data folder and then my data is synced everywhere, woo!"
  10 [18:03] <aquarius> This is great, until it isn't.
  11 [18:04] <aquarius> If you make changes to your SQLite database on two different machines, they'll conflict, because SQLite isn't designed for syncing.
  12 [18:04] <aquarius> So you'll get a conflict from Ubuntu One file sync, which you'll have to resolve.
  13 [18:04] <aquarius> U1DB is cleverer than that; changes to your data won't conflict unless they really, really need to, so it's much more suitable for syncing.
  14 [18:04] <aquarius> You probably have questions at this point, but let's see some code first and then we can discuss.
  15 [18:04] <aquarius> First, you'll need u1db, of course.
  16 [18:05] <aquarius> For this talk, you'll need to get u1db from launchpad, with "bzr branch lp:u1db"
  17 [18:05] <aquarius> (U1DB itself is now available in Ubuntu Quantal, but for this demo please get the Launchpad version because it contains the example app as well!)
  18 [18:05] <aquarius> Once you have u1db, this should work and not throw errors: PYTHONPATH=u1db python -c "import u1db"
  19 [18:05] <aquarius> (I pause a little to give people a chance to grab u1db :))
  20 [18:06] <aquarius> Let's try a simple example of a working app first: the u1db distribution comes with an example app called "cosas", which is a small todo list.
  21 [18:07] <aquarius> (Everyone writes a todo list now: it's like the Hello World of the 21st century)
  22 [18:07] <aquarius> cd u1db/cosas
  23 [18:07] <aquarius> PYTHONPATH=.. python ui.py
  24 [18:07] <aquarius> and you should see a todo list window pop up, looking like http://ubuntuone.com/7aOvtpIljWwbwB1FEFbs5L
  25 [18:08] <aquarius> (you might need to apt-get install python-qt4 if you don't have it)
  26 [18:08] <ClassBot> calmi asked: python is the recommended language?
  27 [18:09] <aquarius> calmi: I'll get to that. It is *a* language you can use :)
  28 [18:09] <aquarius> there are many others :)
  29 [18:09] <aquarius> Add a couple of tasks, tick a couple
  30 [18:09] <aquarius> Then from the app menu, do File > Synchronize
  31 [18:09] <aquarius> and choose Ubuntu One and Synchronize Now
  32 [18:09] <aquarius> (this demo is for people who have an Ubuntu One account; if you don't have one, just skip doing this and take my word for it. You can sync u1db without using Ubuntu One at all by running your own server)
  33 [18:10] <aquarius> Your little todo list is now synced with your U1 account!
  34 [18:10] <aquarius> You can prove this: quit cosas and then delete the file ~/.local/share/cosas/cosas.u1db, as if you're a second machine
  35 [18:11] <aquarius> If you now restart cosas you'll have no todo list items... just File > Synchronize again and they'll be synced to you!
  36 [18:11] <aquarius> Obviously you could be running cosas on many different Ubuntu machines and the data could be synced to all of them.
  37 [18:12] <aquarius> So, let's see how this actually works. I'll show using U1DB from Python on Ubuntu, but it's also available in other platforms and languages too, which I'll talk about later.
  38 [18:12] <aquarius> Start with some simple examples, taken from the documentation at http://packages.python.org/u1db/
  39 [18:12] <aquarius> Start a python interpreter with "python"
  40 [18:12] <aquarius> >>> import u1db
  41 [18:12] <aquarius> >>> db = u1db.open("mydb.u1db", create=True)
  42 [18:13] <aquarius> We've now created a U1DB database named mydb.u1db.
  43 [18:13] <aquarius> Next, we'll create a document in it. U1DB is a document-based database: you save JSON documents into it. So, a simple document naming a person:
  44 [18:13] <aquarius> >>> content = {"name": "Alan Hansen"}
  45 [18:13] <aquarius> >>> doc = db.create_doc(content)
  46 [18:13] <aquarius> And the Document is saved in the database. You can still see the Document's content:
  47 [18:14] <aquarius> >>> doc.content
  48 [18:14] <aquarius> {'name': 'Alan Hansen'}
  49 [18:14] <aquarius> We can edit the content of that document, of course:
  50 [18:14] <aquarius> >>> doc.content = {"name": "Alan Hansen", "position": "defence"}
  51 [18:14] <aquarius> After changing the content, we need to save that updated Document:
  52 [18:14] <aquarius> >>> rev = db.put_doc(doc)
  53 [18:14] <aquarius> And now the updated document is saved in the DB, ready to be retrieved or queried or synced.
  54 [18:15] <aquarius> Let's create a couple more documents:
  55 [18:15] <aquarius> >>> doc2 = db.create_doc({"name": "John Barnes", "position": "defence"})
  56 [18:15] <aquarius> (and we'll change the content before saving: Document.content is a dictionary)
  57 [18:15] <aquarius> >>> doc2.content["position"] = "forward"
  58 [18:15] <aquarius> >>> db.put_doc(doc2)
  59 [18:15] <aquarius> >>> doc3 = db.create_doc({"name": "Ian Rush", "position": "forward"})
  60 [18:15] <aquarius> and we now have three documents.
  61 [18:15] <aquarius> Retrieving documents from the database with a query is done by creating an "index".
  62 [18:15] <aquarius> To create an index, give it a name, and the field(s) in the document that you want to query on:
  63 [18:15] <aquarius> >>> db.create_index("by-position", "position") # create an index by passing a field name
  64 [18:16] <aquarius> so we have an index, named "by-position", indexing all documents on the field "position".
  65 [18:16] <aquarius> And now we can query that index for a particular value. If we want to get everyone with position="forward" in our list of people:
  66 [18:16] <aquarius> >>> results = db.get_from_index("by-position", "forward")
  67 [18:16] <aquarius> And our results is a list of two Documents:
  68 [18:16] <aquarius> >>> len(results)
  69 [18:16] <aquarius> 2
  70 [18:17] <aquarius> And you can manipulate that list just like a standard Python list, which is what it is:
  71 [18:17] <aquarius> >>> data = [result.content for result in results]
  72 [18:17] <aquarius> >>> names = [item["name"] for item in data]
  73 [18:17] <aquarius> >>> sorted(names)
  74 [18:17] <aquarius> [u'Ian Rush', u'John Barnes']
  75 [18:17] <aquarius> That's the very basics of using u1db: saving data, loading it, and querying for it, same as any database.
  76 [18:17] <aquarius> The documentation at http://packages.python.org/u1db/ goes into much, much more detail.
  77 [18:18] <aquarius> In particular, the tutorial at http://packages.python.org/u1db/tutorial.html walks through cosas, the example todo list app, and explains how it structures its data and how to sync a U1DB with other places, like Ubuntu One.
  78 [18:19] <aquarius> As I said right at the beginning, U1DB is designed to work everywhere.
  79 [18:19] <aquarius> This means that there will be, or could be, a U1DB implementation for any choice of platform and language.
  80 [18:19] <aquarius> What I've shown above is the Python implementation, which should work on any platform where you have Python (so Ubuntu, other Linuxes, Windows, Mac, N9, etc).
  81 [18:20] <aquarius> There is also a C implementation, so if you're writing apps in C or some other C-bound language you can use the C version.
  82 [18:20] <aquarius> At U1 we're also building an Android Java version and an iOS Objective-C version, in time.
  83 [18:20] <aquarius> There's also a command line client (u1db-client, and u1db-serve to run a simple server).
  84 [18:21] <aquarius> Members of the U1DB team are also bringing U1DB to Vala, Go, and in-browser JavaScript.
  85 [18:21] <aquarius> So apps using all those languages on all those platforms will be able to sync data: imagine your app on Ubuntu and a mobile version on your Android phone and a web version, all able to sync data between themselves.
  86 [18:22] <aquarius> Part of the goal of U1DB is that if you decide that you want to be able to use it in a platform we haven't provided (and you can't or won't bind to the C version) then it ought to be possible to do a full reimplementation of it in your chosen language for your chosen platform
  87 [18:23] <aquarius> There's a very comprehensive test suite exactly so that it's possible to test compliance.
  88 [18:23] <aquarius> So we're building U1DB for a whole load of different platforms, and it's possible to bring it to others.
  89 [18:23] <aquarius> U1DB is in Ubuntu Quantal, so your apps can depend on it.
  90 [18:24] <aquarius> You don't actually have to sync data to use U1DB, of course: you can just use it as a database and never sync it at all!
  91 [18:24] <aquarius> If you use U1DB in an app now, then you can add syncing later when you choose to.
  92 [18:24] <aquarius> This has been a brief tour of U1DB, and I'm sure there are questions, so I'll happily take them now :)
  93 [18:24] <ClassBot> jsjgruber-l85-p asked: How is this different from couchdb?
  94 [18:25] <aquarius> Similar goals, similar-ish implementation, but the big difference is that U1DB runs in your process, and it's built separately for each platform and uses the platform's native data storage capabilities.
  95 [18:25] <aquarius> So couchdb was a server; if you wanted to use couchdb on your phone, you (or someone else) had to port the whole couchdb server to that platform, and run it as a server there and connect to it.
  96 [18:26] <aquarius> u1db is built separately and independently for each platform
  97 [18:26] <aquarius> so it's not making one codebase run everywhere; an implementation can do whatever makes the most sense on the platform it's on: an Android version would likely be written in Java, for example
  98 [18:27] <aquarius> and an in-browser JavaScript version would be run entirely by your web app; it's not a separate server that you have to deploy and secure, it's entirely client-side and then you sync it with a u1db server elsewhere.
  99 [18:27] <aquarius> using the DB is similar: it's a store of JSON documents, which is similar to couch.
 100 [18:28] <aquarius> does that answer the question? :)
 101 [18:28] <ClassBot> jsjgruber-l85-p asked: How will it avoid the scaling problems couchdb had on the U1 infrastructure?
 102 [18:29] <aquarius> we've deliberately built the U1DB server infrastructure to work exactly for this use case
 103 [18:30] <aquarius> I'm not the server scaling expert, but I know the people who are :) If you have specific questions, we can answer them
 104 [18:31] <ClassBot> wan26 asked: What encryption scheme is used for sync?
 105 [18:31] <aquarius> Sync to Ubuntu One is always over SSL
 106 [18:32] <aquarius> If you run your own server, you are able to run it over SSL or not as you choose
 107 [18:33] <aquarius> and sync to U1 is authenticated the same way as all the other Ubuntu One APIs
 108 [18:33] <aquarius> again, if you run your own server, you can (and should) define your own authentication policy :)
 109 [18:34] <aquarius> The best place to learn about u1db is the documentation, as mentioned, at http://packages.python.org/u1db/
 110 [18:35] <aquarius> There are introductory materials there to get started, and a much deeper dive into the intricacies of querying, syncing, replication, and so on when you want to do more complicated things
 111 [18:36] <aquarius> or if you want to do a new implementation of u1db for a different platform
 112 [18:36] <aquarius> but the best thing is to use u1db as it exists and build apps on it :)
 113 [18:38] <aquarius> one of the apps in the recent Ubuntu App Showdown (tickit) is using u1db for storage precisely to get syncing capabilities
 114 [18:38] <aquarius> and we have a bunch of others talking to us as well :)
 115 [18:38] <aquarius> you can get hold of the u1db team if you have questions once you start development on irc, here on freenode, on #u1db
 116 [18:39] <aquarius> and the mailing list at https://launchpad.net/~u1db-discuss
 117 [18:40] <aquarius> and if you fancy helping me work on the JavaScript version, that'd be great :P
 118 [18:43] <aquarius> So, that concludes the brief tour: if you have questions, or you're thinking of using u1db, or curious as to how you'd use it, speak now. :)
 119 [18:45] <aquarius> Cool, no more questions
 120 [18:45] <aquarius> So, thank you for your time: come and tell us about which apps you're using u1db in!