CRIU

Now, when the 0.3 is out, a few words about our next plans.First of all, it looks like criu releases are better to be done soon after some Linux kernel releases. This is because the criu project actively patches the kernel, and the new features we ... [More] add heavily depend on the kernel API changes, that appear with the most fresh kernel. Thus making the criu release before any kernel is not convenient, and doing it long after it makes little sense -- it's better to wait a bit for the next kernel and get even more API updates and more features working on the upstream kernel.That said, the next 0.4 release is planned to be soon after the Linux 3.8. As was said, we expect to have the ipc, inotify-epoll-etc and packet sockets stuff merged there.As far as the "future changelog" is concerned. First of all, we will definitely do a lot of bug-fixing. So far the project sub-goal has been to define what kernel changes are required to make this all possible. Now this rush is more or less over and we will pay much more attention to the code stability. All the more so the crtools users community is now bigger than just the OpenVZ team :)Other than this, we have several new features which already exist in the form of patches. They are -- supporting tasks' FPU state, shared mm/files/etc stuff, fanotify, tasks' and threads' pending signals, file locks, posix cpu timers and (wow) the ARM port!Challenging, isn't it? Hopefully, all these plans will get carried out! And we'll keep you updated as things go. [Less]

One more real software checked -- now it's the tcpdump tool!Since tcpdump is typically launched from shell, the --shell-job option should be used. Another thing to note is that it requires a certain support from the kernel, but the good news is that all this support now sits in the net-next git tree, and thus will appear in Linux-v3.8.

We've just presented CRIU on LinuxCon Europe! The slides are available!Download ODP: http://download.openvz.org/~xemul/criu/criu-lce2012.odpView online: http://www.slideshare.net/kolyshkin/checkpointrestore-are-we-there-yet

Houston, we have had a mailing list subscription problemIt has been discovered that our link on http://criu.org/ wiki (which redirects to a mailing list subscription web-form) was broken, thus if you've been trying to subscribe to CRIU-ML and didn't succeed -- now it's a time to make another attempt.We're deeply sorry for inconvenience.

A new feature in crtools! It wasn't in plans, just appeared to be easy-to-implement and useful while working on memory snapshot. The feature is -- an ability to execute a system call from another task's context.It does exactly what is written -- you ... [More] can take some task and make it execute a system call you want with arguments you want. Then get the syscall result printed on the screen. Currently, crtools only allows you to effectively execute system calls that do not accept arguments (e.g. extremely useful getpid()), or accept those that are either plain numbers or point to some strings in memory (e.g. close() or open()). But even with this little support it's already possible to do interesting things, like re-opening file descriptors, tuning VM bits with madvise, altering mappings, changing session/group, etc.In the future, there will appear the ability to pass a pre-allocated buffer into a system call and dump its contents on the screen after it (e.g. to call read() effectively). Another thing, that should appear (but it's not yet clear how to implement it) is ability to describe some binary structure and push one into a system call (e.g. -- sigaction() and sigprocmask() accept pointers to memory buffers, that are not strings).This feature should work on kernels starting from 3.3, since it doesn't require all the new stuff we've added, but only needs the parasite-code-injection facility that appeared upstream long time ago.Try it out! And if it complains about system call you want being unknown (it will) -- let us know :) [Less]

One more plan has turned into the code -- tasks' FPU state is now (hopefully) properly handled by C/R!On dump crtools fetch the FPU registers out of the kernel using the existing ptrace API. Restore is done without the ptrace, although it's tempting ... [More] to do so. Instead, we restore the final pieces of task's state using the sigreturn system call, it allows us to effectively debug the restoring process with strace and gdb. That said, FPU state is restored by providing the properly cooked FPU frame to the sigreturn system call.So, if you're playing with FPU instructions in your application and wanted to try CRIU out -- it's time to do it! [Less]

You are probably already well aware that CRIU is mostly implemented in userspace. We even put this into its name: Сheckpoint/Restore In Userspace. So, we need to test this userspace stuff we develop. A very important decision made at the early stages ... [More] of the project -- every feature and a piece of functionality should be covered by a test. So, in parallel with criu tools we are developing a test suite for it, which is actively used to find bugs and regressions.But is this test suite any good? There are many approaches to answer this question. One particular approach is to use code coverage tool (gcov) to find out how much code is covered by tests, i.e. what code is executed and what is not when we run these tests.So, a few days ago +Andrey Wagin went ahead and added the needed machinery to crtools (see http://git.criu.org/?p=crtools.git;a=commitdiff;h=134937bc). Note that gcov generates plain text files with stats, so Andrey also used lcov to convert that to bright colored HTML.The results are surprisingly good, showing our test suite in its the full glory. See it for yourself, link is below. It looks like it was a very wise decision to develop a test suite in parallel with the software. [Less]

Yet another feature that will appear in v0.4 -- shared file descriptors table support.Different tasks may share various resources. For example, when a task calls open() and the fork()-s the file, referenced by the obtained file descriptor becomes ... [More] shared between the parent and the child. Shared means that modification of the file state (e.g. changing the file position implicitly with read or write or explicitly with lseek) made by either task are seen by the other.In similar situation can occur not only an individual file, but the whole table with file descriptors. In this case changing the table with open() or close() (or lots of other system calls) made by one task will be seen by the other. This is typically observed in multi-threaded tasks, but even different tasks may have an fd table shared -- there's a CLONE_FILES flag that can be passed to the clone() system call that makes child share fd table with parent.What it means for CRIU? As usually two things. 1st, we should be able to detect this situation. Fortunately, this was solved long time ago already with the kcmp() system call. Just like we did it with shared files, now we compare fd tables from different tasks to each other and find out which of them are shared.2nd, we should be able to restore this state. This part is both easy and complex. It's easy because the kernel already has an API for creating two tasks with shared fd table -- the mentioned clone() system call plus CLONE_FILES flag. But it's complex because once we create two tasks with shared fd table we can no longer easily toss file descriptors in both tasks, in particular -- restoring the fdtable contents (i.e. -- reopening files sitting in it) should be carefully synchronized with all the other tasks that share fd table and open/close descriptors for their restoration needs.This has been solved! But, I cannot easily describe here how it was done, sorry, because it's pretty much technical thing. We will try to describe the changes in the "under-the-hood" section on our wiki.Next potentially shared object that we're about to handle is -- VM. Tasks can share one using the CLONE_VM flag to clone. This thing will be implemented in the similar manner, but with even more precautions on the restore part, as once cloned a child with shared VM cannot even modify a single variable without risking to affect its parent. And this is a great challenge! [Less]

Good news, everyone! One more feature developed and is going to work out-of-the-box in v0.4. It's Linux-specific (though widely used by software) files such as inotify, fanotify, signalfd, etc. It's a very long story, which began about 9 months ago ... [More] and now it's technically over, we just need to wait a little bit more for official kernel and crtools releases.So, Linux kernel provides a very interesting set of APIs -- a set of system calls, that create a file descriptor that can later be used for things which are not sending or receiving data streams. These are: inotify/fanotify files that can generate events about filesystem objects (inodes or mount points) state changes; signalfd that can be used to poll() and read() signals sent to a process; eventfd file that is used to poll() for an "abstract" event represented by an increasing integer and some other. All these files could not be easily supported in CRIU for standard reason -- kernel provided no information about the required internal state of them. For example for signalfd we need to know the mask of signals it serves, for eventfd -- the event counter value, for inotify -- information about which inode is being watched and how etc. In order to get this information we had to extend the kernel API, and on LSF conference last year the community judged that the suitable API for this might be the /proc//fdinfo/ files. At that time these contained only basic information about opened files -- position and flags.We extended the contents of these proc files, so that every file descriptor could show any additional information about itself in proc. Thus the signalfd file shows the mentioned mask, the eventfd -- the event counter, inotify/fanotify files -- information about inodes being watched, etc. This set was discussed on the mailing lists for more than half a year (!) before it was finally merged by Andrew Morton into -mm tree. Then it was sent to Linus and now it sits in the v3.8-rc queue waiting to be released some time soon.Another part of the feature was crtools code. The support for simple files such as signalfd and eventfd was developed soon after the LSF and this made it possible to checkpoint and restore Apache, MySQL, cron daemons and some other software. The only missing part was support for less popular files -- fanotify. Now it's done with the epic patch set from +Cyrill Gorcunov.That said, once Linux-v3.8 and crtools-v0.4 is released, A LOT of Linux software will be supported by crtools withing the need in custom kernel! [Less]

Today is the first day of the CRIU developers community growth!So far the tools part of the project was developed by developers from Parallels / +OpenVZ. We've had a couple of patches from other people, but these were rare and actually were cleanups. ... [More] This time we have the whole new feature -- Huawei people developed and contributed the support for file locks (both flock and posix)!More features is great, but more contributors is awesome! [Less]