XMMS2

In a previous post, I presented some components that were built from scratch and which formed the foundation of a new client for XMMS2. This client is geared toward handheld devices and is being developed as part of Google Summer of Code 2010. After ... [More] a month of hacking I was able to make good progress on the code and we now have a first version of the app ready for testing. I made a little demo video of the app. Oritide : XMMS2 mobile client from allad on Vimeo. Here is quick overview of the features supported at the moment: Now Playing screen play/pause, previous, next command volume control playback position control cover art support now playing songs list with tap support for playing any of them playlist shuffling Playlists screen list all non-hidden playlists of the daemon support for playing any song of any playlist at any time Media library browsing library browsing by artist > album > songs quick filter for easily finding artist artist’s albums playback support Search screen support for finding songs from artist, album or title queries direct playback of any song The new client is called Oritide and is being developed as web application. So if you have a modern browser (with good support for HTML5 specifications) you are good to go.  The client has been successfully tested on iOS devices as well as Android handsets. You can download the beta application here : http://www.megaupload.com/?d=8D13UQBQ I look forward to getting your feedback. [Less]

As part of GSoC 2010 I’ve been working on a new mobile client for XMMS2. It is a web app that can be used to remotely control XMMS2 playback. This web app consists of two parts. First, a python script which exposes an HTTP server and acts as a bridge ... [More] to the XMMS2 daemon. Second, the actual web app which is gonna use the Sproutcore framework. We’ve decided to use the Sproutcore framework to build the web UI because it gives us great flexibility. Sproutcore has a very good rendering layer which can be used to build web equivalent of virtually any widget. Plus Sproutcore, in its latest devolpment version has very good support for touch enabled devices. Here is a mockup of what we want the interface to look like . The goal is to use the latest web technologies to design an app with a native look-and-feel. In order to achieve that,  I’ve identified 3 core components that are needed and will be reused throughout the app: an iPhone-like tab control a iPhone-like list view a navigation component that offers push/pop like views management For the past month I’ve been working on them  and we now have working and functional implementations. There’s a live demo of the tab control and navigation system at the following address : http://cherif.ycsoft.org/static/xmms/en/1b1/ Below are two captures of these components : Tab Control and navigation bar List view component Please note that the list view is not present on the previous demo link. This is due to some development limitations. However you can check out the source for the components using git : git://git.xmms.se/xmms2/x2web-allad.git/ or view it online at x2web-allad.git These components have been packaged under in a framework called Iweb. (framework is Sproutcore lingo for libraries) and will be later contributed back to the open source Sproutcore project. Now that we have these, work has started on the actual client web application. Stay tuned for some more updates soon [Less]

It turns out Erik Massop (nesciens) had some code to speedup SQL queries in his repository. Here it is compared to the old SQL code and S4 midb: Query SQL Old >SQL Experiment S4 midb Avg (µs) S (µs) Avg (µs) S (µs) Avg (µs) S (µs) “one” ... [More] 180115.3 ~5521.6 326.0 ~8.9 127.7 ~3.7 “*” 365540.5 ~7049.7 2232619.0 ~18016.8 148165.8 ~3209.5 “artist:metallica” 6864.2 ~214.3 4239.5 ~11.4 3336.5 ~219.5 “tracknr>20″ 30378.6 ~789.9 25947.6 ~466.0 1871.7 ~132.7 “tracknr>30″ 27624.5 ~1372.5 25409.9 ~1502.8 236.1 ~7.5 “+comment” 238729.9 ~7028.3 73566.2 ~1553.5 58894.9 ~2003.7 “tracknr:4″ OR “artist~foo” AND NOT “artist:weezer” 188794.7 ~4773.8 49245.6 ~1210.9 18785.8 ~638.8 As we can see most queries are getting a nice speedup. Most notably “one” which is about 500 (!) times faster. But also “+comment” (3.2 times faster) and that long last query (3.8 times faster) gets a nice boost. The only one getting slower is “*” (taking over 2 seconds). S4 is still faster than SQL, but all the SQL queries (with the exception of “*”) are under 100 milliseconds on a moderately sized media library (~10,000 songs). [Less]

I finally had time to run a few benchmarks. I wanted to measure how the different back-ends compared to each other in query performance, memory consumption and disk usage. The back-ends being compared are SQLite (the current back-end used in ... [More] xmms2-devel), S4 mmap (the “old” S4 back-end) and S4 midb (a new experimental S4 back-end). Just to cover my back: I do not have much experience with benchmarks, and I am in no way an objective observer (slight bias towards S4, me?). If you see something that seems wrong, let me know; I probably made a mistake. Query performance First we will look at query performance. This is perhaps the factor that is most visible to the end user. The time measured is the time spent inside the xmms_collection_client_query_infos. It was measured by running gettimeofday at the beginning and end of the function and printing the difference in microseconds. I also used a slightly modified nyxmms2 that uses xmms_coll_query_infos instead of xmms_coll_query_ids. So, to the results. The table shows the average time used in the query (calculated by where is the time used on the i’th run) and the sample standard deviation (calculated by and then taking the square-root to get S). n was 10 for all benchmarks. The last column is the average time that SQLite used divided by the average time S4 midb used. Query SQLite S4 mmap S4 midb SQLite / midb Avg (µs) S (µs) Avg (µs) S (µs) Avg (µs) S (µs) “one” 180115.3 ~5521.6 203.2 ~5.2 127.7 ~3.7 ~1410 “*” 365540.5 ~7049.7 286972.0 ~5489.7 148165.8 ~3209.5 ~2.47 “artist:metallica” 6864.2 ~214.3 5377.6 ~175.7 3336.5 ~219.5 ~2.06 “tracknr>20″ 30378.6 ~789.9 2885.1 ~89.4 1871.7 ~132.7 ~16.23 “tracknr>30″ 27624.5 ~1372.5 454.7 ~77.0 236.1 ~7.5 ~117.0 “+comment” 238729.9 ~7028.3 106268.0 ~3277.4 58894.9 ~2003.7 ~4.05 “tracknr:4″ OR “artist~foo” AND NOT “artist:weezer” 188794.7 ~4773.8 26407.7 ~793.9 18785.8 ~638.8 ~10.05 As we can see S4 is faster in all cases, ranging from about 2 to over a 1000 times faster. Why “one” is so much slower on SQLite I don’t know, could be inefficient SQL query, could be something else. Also worth noting is that S4 is mostly bound by the size of the result set. Queries resulting in small result sets (“one” for example) gives short query times. I’ll get back to why later. Memory consumption To get good performance S4 trades memory for speed. The big question is if the trade-off is worth it. For now we will settle for finding out exactly how much memory the different back-ends use. Memory consumption was measured by using massif, a valgrind tool. The back-ends were run two times, once by just starting up and shutting down and another time with a query for “*” before shutting down. That way I hope to visualize idle memory usage versus usage when searching. All numbers are in MiB. SQLite S4 mmap S4 midb Idle Search Idle Search Idle Search 10.43 17.77 1.13 10.59 23.79 32.56 As we can see both S4 implementation adds about 9 MiB when searching compared to the 7 MiB SQLite add. Also it may seem like S4 with mmap is the big winner here, but what massif does not show is shared memory, the kind mmap uses. If that had been counted in we would have added about 22 MiB to S4 mmap, bringing it up to about the same memory usage as S4 midb. So with a media library with 9,361 entries S4 uses a little over twice the memory SQLite uses at idle. Disk usage Finally, the one most people probably will not care about with today’s abundance of disk space: disk usage. It’s simply the size of the datafile. SQLite S4 mmap S4 midb 41.5 MiB 21.7 MiB 5.4 MiB Summary In terms of performance and disk space S4 (especially the midb version) is the clear winner, but it eats up about twice as much memory compared to SQLite. Fair tradeoff? Maybe for a 10,000 entry media library, but what about one with 100,000 entries? We would probably see memory usage around 200 MiB. A note on S4 query performance As I said earlier S4′s query speed is mostly bound by the size of the result set. To see why we have to dig a little. Running XMMS2 in Callgrind (another Valgrind tool) is a nice way to find bottlenecks and hotspots. Opening the generated callgrind.out file in KCachegrind reveals a few interesting things: As we can see xmms_medialib_query_ids (the function calling s4_query) only contribute 3.64% of the running time of the query. The rest of the time is spent fetching the properties (artist, album, …) of the entries we found in the query and inserting them into the dictionary we return. We see that _entry_propery_get_ is called 28,000 times. The media library this was run on has about 500,000 relationships (a relationship is an entry-to-entry mapping, for example (“songid”, 1) -> (“title”, “One”)). Using B+ leaves with room for 101 entries and a fill rate of 50% (it was measured to be around 53%, a bit away from the normally assumed 67% for B+ trees) this gives about 10,000 B+ leaves. A simple traversal of the 10,000 leaves would probably be faster than calling _entry_property_get_ 28,000 times. An observation well worth taking into account when we design the S4 query layer. [Less]

Summer is here, and we all know what that means: GSoC and hack time! This year I will try to document the progress here. Who am I? I am a 22 year old computer science student from Norway attending GSoC for the second time. Once again I was given a ... [More] chance to work on XMMS2 and S4. It is off to a slow start at first because of some exams, but after June 4th it is full speed ahead. As you may know my project this year involves developing a new database backend called S4 for XMMS2. The project will be mentored by Sébastien Cevey. Some of goals of this year’s project is Proper support for source preferences A standalone S4 query layer Coll 2.0 support To keep track of time we have set up some milestones with associated deadlines. The first one coming up at June 6th is to have a benchmark set up so we can track performance against the old database, but also against itself as we do changes to see if they are worth it. I cheated a little and started early. The last week I have been trying a new approach in S4. Instead of storing the index data structures on disk as before we build them on startup. The start up takes a little longer (about 0.5 seconds on a media library with 10,000 entries), but otherwise it seems to perform better or on par with the old code using mmap. With this new approach we will hopefully get a more reliable database, smaller on disk file and some performance improvements. A more thorough analysis will be provided when I have time to run some proper benchmarks. Happy hacking everyone! [Less]

Just a brief warning to anyone looking for the XMMS1 source code. Please don't use the code at http://xmms.org since the domain is no longer controlled by the people in the community. Please use http://legacy.xmms2.org instead. For background ... [More] information you can see the following post by Tobias: http://thieta.wordpress.com/2010/04/28/what-ever-happened-to-xmms-org/ [Less]

Today Google announced the students accepted for this year's Summer of Code. We received four slots this year, and selected some really great sounding projects this year.Sivert Berg (cippo) - Mergeable S4 with Coll 2.0 mentored by Sebastien ... [More] CeveyKonrad Scorciapino (konr) - GIMME - GIMME Interesting Music, Mr. Emacs! mentored by Alexander Botero-LowryBruno Ronzani (Bruno') - Crossfade plugin & co mentored by Anders WaldenborgCherif Yaya (allad) - XMMS2 mobile web client for iPhone and Android devices mentored by Daniel SvenssonIn the past we have not had a heavy focus on client development during the Summer of Code, but this year we were happy to see a number of proposals that allowed us to split the selection fifty-fifty between client development and infrastructure work. We are also very happy to have Sivert Berg back to continue his work on s4. We've already seen excellent performance results from initial looks at integrating s4, so we expect with a bit of polish and close work with Sebastien and Erik to assure good integration with Collections 2.0, that XMMS2 will have a great new database backend ready to go for users! [Less]

This blog is now deprecated, see http://thieta.wordpress.com for new updates.

Permalink to my proposal for Google’s Summer of Code 2010. – I’ll develop GIMME, a client for xmms2 on Emacs that puts an interestingly strong weight on searching and (mis)using collections as search results. The goal is quickly finding out what the ... [More] user feels like playing. 2 mpd, xmms2, motivation and filter-view The first time I heard about XMMS2 was on #emacs when I was bragging about how ncmpcpp was the best player ever. Some guy mentioned that xmms2 was another player like mpd, and I kept that in mind. Now wednesday night, I was casually browsing GSoC’s page and I’ve found out that idea of an xmms2 client on emacs! Now what could be more awesome than this, than developing a full package for emacs on SoC? But I’m an mpd guy (a.k.a. the enemy), right? Well, I love mpd, but what I love about it is its client/server architecture and awesome clients, so it can be said too that I’m also a lover of xmms2! I loved mpd’s players, particularly ncmpcpp, because I used to have all my songs in the playlist, and filter them as a way to browse and find out what I felt like playing. I liked that approach because while narrowing the collection, I’d usually change my mind and find something more interesting along the way. Still, it was not perfect as I could only filter once, and the kept lines weren’t really related to each other, so I had to manually queue the tracks. GIMME’s interface will be entirely based on this experience and on ways of improving it. It with feature three views, one of which is the filter view, where you start with your whole collection (or part of it, for performance, as long as it gives you the impression of totality), and narrow it until you find a collection of songs you’d like to play, Here is a use case to illustrate. Keep in mind that the interface will have colors and perhaps a menu, and will be much prettier. The user opens the filter-view. “Oh well, what shall I play today? Beethoven? Classical music!” The user filters ’style:classical’. “Ah, actually I feel like playing Bach” The user filters ‘Bach’. “But what of bach? The canons from the goldberg variations! The user filters ‘Canone’Now the user can choose to (o)pen selected song, (a)ppend it to the playlist, (O)pen the collection, sorting it, (A)ppend the ordered collection. 3 Tree-view There will be a history of these collections/searches, and key-bindings to make cycling through them easier. For example, the user can go back to step 1 in the example and filter Beethoven. This will turn the search history into some sort of tree. There will be key-bindings to make this easier on the filter-view. Perhaps we can use the numbers [1-9] to go back to previous steps. Still, there will be a view in which it will be easy to compare various search trees, and it will be called “tree-view”. Tree-view’s a trivial view of the filter history, displayed like a nested list on org-mode, and with many key-bindings to make cycling easy. My main focus is usability. Here is a sketch of the previous example. 4 Playlist-view and implementation Just a regular playlist… My idea is to build this part from vanicat’s lagn, making it nicer. I will also create my own client to communicate with the elisp part, instead of using nyxmms. To be brief, the problem with it is that I won’t be able to stream query results to a process filter, something that I’ve witnessed to dramatically improved the performance of the “notmuch” application, and I won’t be receiving notifications, burdening the Elisp part. It would be uber-leet to implement the IPC on elisp, but I read it somewhere in the wiki it was not a good idea, and I fear it would take much time, comparatively to the client approach, which, after seeing an example and the xmms-client tutorial, seems pretty easy to code. 5 Cool ideas These three views make the core of GIMME, yet some other features are also cool, and will be added if I have enough time remaining. 5.1 Bookmarks-view It’d be interesting to create a view with “bookmarked” collections, which could have dynamically created collections, too! Imagine having a collection of your most played songs – If it is dynamic, it will change every time you see it. 5.2 Status-bar I’m not sure if it’s a good or a terrible idea, but integrating GIMME’s status on emacs’s status bar would have some use, at least to me. 5.3 Lyrics Another feature I like about ncmpcpp is that it fetches the lyrics of the song you are listening to. For classical music, however, this doesn’t work out fine, and since GIMME will be quite reprogrammable, the user will be able to write his own algorithm to fetch the lyrics of a particular kind of song. 5.4 Add the whole piece In some genres of metal, noise and on classical music, a single song has many parts (Allegro, Adagio), and it would be nice to group them together in some situations. Imagine you like that you like the softy part of the moonlight sonata so much that it is your most played song. You could then, when on the ‘most played’ collection, play the (E)ntire piece with one button. 5.5 Add it to Emacs’s package manager Thus making our users’s lives easier. 5.6 Omit repetitions It would be a cool to be able to get an overview of the totality of your songs, instead of seeing only the the ones that happen to be the first. I could implement some way of omitting songs that don’t add much to the question “what should I play?”. Perhaps by grouping, like “31 – XXX Andante, sempre cantabile.mp3 (and 30+ from Beethoven)” 5.7 Easier search It would be nice to be able to search for “a:Ma Ma” instead of “author:Marilyn Manson” 5.8 Amor vincit omnia Imagine sharing your songs on Emacs directly with someone who is on IRC, using ERC? I don’t know if it’s possible, but a nice long-term plan for the project! 6 Time-line 3 months = 12 weeks. Weeks 1-2: Building a basic controller interface Weeks 3-6: Working on filter-view Weeks 7-10: Working on tree-view Weeks 11-12: Working on other stuff 7 Who is this konr guy? I’m Konrad Scorciapino, from the University of Campinas, Brazil. My nickname on freenode is konr and my e-mail is [email protected]. I’m in my last year of Computer Engineering, and I currently work on my own (I think the correct term is freelancer, but it sounds so depressingly undesirable…) in a Computer Vision project. I’ve also programmed quite a bit on college, including projects in a team environment. I’m a somewhat decent Lisp programmer, being Clojure my main language, but I know pretty well C, PHP, Java, Elisp, CL, some Python, and the web stuff. I’ve also inflicted much pain and suffering upon myself to learn some visual tool-kits (mostly QT and Swing), which perhaps will be valuable skills to the project later. I haven’t coded in any serious Free Software project, but I’m pretty familiar with the tools generally used (git, Emacs, vi, lein etc) and I’ve helped with some translations on the Fluxbox and FreeBSD (this one a looong time ago) projects. I’m also an enthusiastic Emacs user that hacked a lot of plug-ins, most notably viper and should someday organize the changes and commit them. My University is also considered one of the best in my country (chupa Poli), so this too adds a little bit to my ethos [Less]

Again, XMMS2 has been accepted as a mentoring organization \o/. As you probably already know, if you are reading this blog, last year I participated as a student for XMMS2 in the awesome GUI client project, I can’t say it was very successful and that ... [More] I have fulfilled my expectations,  I won’t discuss the reasons here, but I can say that the fact that it is a very big contributed. My work was basically related to prototyping and experiment with different ideas trying to find the best ones. This year there are two independent sub-projects: Collections browser, try new ways browse your media library, find what to play, build collections and more. If you have some crazy, creative ideas on how to do that this is the project to apply. Take a look at last year mockups to get the feel of it, as you will probably have to do some too. Column browser mockup Infrastructure, in this project you will be doing the base code for the client. Will it be based in plugins? How new elements can be added to it? Communication between GUI elements? Use QML, QtScript? Lots of questions and coding to do. In my opinion both are interesting with lots of coding, thinking and ideas. The collections browser probably being the most creative one and the infrastructure project for the ones that want to build the awesome base for the awesome client. I recommend everyone to read theefer’s Star Wars series of posts about the awesome GUI client and if you are thinking of applying for one of the related GSoC projects  they are a must read. Ep. IV: A New Hope Ep. V: The Flamewar Strikes Back Ep. VII: Return of the GUI Ep. I: The Fandom Menace If your not much into GUI, there are a lot of other interesting ideas for GSoC projects at the xmms2 wiki, take a look, go to IRC, discuss and apply. Now, if you are already reading this post, you are probably interested, so, follow the steps (if you haven’t already): install xmms2 git clone git://git.xmms.se/xmms2/xmms2-devel.git git clone git://git.xmms.se/xmms2/calypso.git go to #xmms2 @ freenode and ask about GSoC Some great ideas were discussed and proposed during last year GSoC, unfortunately, not many implemented. I will be doing another post summarizing last year’s GSoC, hoping it will help this year students. Happy (summer) coding! [Less]