An interview with preemptible kernel patch maintainer, Robert Love

Click here to learn
about this Sponsor:

Home | News | Articles | Polls | Forum

Keywords: Match:

Rick Lehrbaum (Jan. 18, 2002)

[Updated 3:00pm PST, Jan. 18, 2002] In this interview, LinuxDevices.com founder and executive editor Rick Lehrbaum chats with Robert Love, the principal maintainer of an increasingly popular kernel-preemption patch that improves the real-time responsiveness of the Linux kernel. Love describes his role in the project, explains why the preemption enhancement is important to a broad range of Linux applications beyond just embedded/real-time (including end-users' desktops), and shares his vision of the future of Linux in the embedded and desktop markets.

RL: You have been becoming well known as the maintainer of a "preemptible kernel patch" for the Linux kernel. How did that situation come about?

Love: I am interested in innovative ideas and the preemptible kernel is a very neat concept. The idea was first talked about during 2.3 (we finally had a kernel that had fine-grained enough locking that it was doable) -- Linus actually outlined the first design. MontaVista released a patch in this timeframe and worked on it through the early days of 2.4.

About this time I tried the patch, rediffed it to the newest kernel, and starting making changes. I eventually posted them. People started trying it and use of the patch took off. I've been working hard since.

RL: What is different between what you have done and what was (or is) being done by MontaVista Software with their preemptible kernel patch? Are you now collaborating with them, and if so, how closely?

Love: The patch I am maintaining now is MontaVista's. They still put time into the patch and I work closely with a couple of their engineers. Obviously it has evolved a lot since I got involved but it's still their work. I believe they are using the latest release in their product.

RL: As I understand it, your interest in the preemptible kernel is for reasons that would tend to be of interest to a broader range of Linux applications such as streaming of audio/video, etc., as opposed to the embedded/real-time system focus of MontaVista. Is this accurate? Can you give some examples of the sorts of applications that your patch can noticeably improve the performance of?

Love: This is correct. I see a preemptible kernel as a means to an overall better system. Besides the traditional markets for low latency -- audio/video, specialized embedded/real-time, etc. -- a preemptive kernel can benefit any interactive task. The result is hopefully a smoother, more responsive, desktop.

RL: With the preemptible kernel patch added, does Linux outperform what the latest versions of MS Windows can offer to desktop users? enterprise users? embedded users?

Love: I've got no idea how we perform compared to Windows. I'd certainly say even earlier versions of Linux were superior to Windows 9x. How we compare to Windows NT (i.e. 2000 and XP) for desktop performance is hard to measure.

The latency of the kernel can certainly be measured -- scheduling latency, jitter, etc. are quantitative -- but the quality of the desktop only begins with the kernel. That means X and everything on top of it factors in, too.

RL: When Red Hat announced that they were going to use RTLinux to provide real-time capabilities to their customers, both Mike Tiemann (CTO of Red Hat) and Alan Cox "went on record" to say that real-time doesn't belong in the kernel. Would you care to comment on that?

Love: Red Hat has an agenda here, and I tend to disregard any technical comments when there is bias. There are benefits to the hard real-time interrupt-driven approach, I don't deny, but that solution is not Linux. I think Linux can become a contender in the embedded/real-time market without giving up on itself, while still being a UNIX and having the standard Linux API. In fact, I think a lot of the technologies that achieve this could live right in the official kernel. Kernel preemption is one such innovation, and it's an innovation that does not benefit solely real-time applications.

RL: Have you gotten any indication from Linus or other key kernel maintainers that your patch (or the MontaVista patch) is being seriously considered for the 2.5 kernel? What do you think the chances are that it will get incorporated into 2.5 or 2.6?

Love: Linus said at ALS this year he was interested in the preempt-kernel patch. That doesn't mean anything to me until we are in, though, but it is a good sign.

There is opposition. There are various issues that need to be dealt with. I believe it is a sane move for 2.5. The patch has seen a lot of testing and we have a lot of users.

I do not want to predict whether it will be merged for 2.5. Time will tell.

RL: Please summarize the advantages in general, not just for embedded real-time apps, of having the preemptible kernel enhancement included in the kernel. What about any disadvantages?

Love: I'll start with a quick explanation of how the patch works. Right now, the kernel is not preemptible. This means that code running in the kernel runs until completion, which is the source of our latency. Although kernel code is well written and regulated, the net result is that we effectively have an unbounded limit on how long we spend in the kernel. Time spent in kernel mode can grow to many hundreds of milliseconds. With some tasks demanding sub-5ms latencies, this non-preemptibility is a problem.

The preemptible kernel patch changes all this. It makes the kernel preemptible, just like userspace. If a higher priority task becomes runnable, the preempt patch will allow it to run. Wherever it is. We can preempt anywhere, subject to SMP (symmetric multi-processing) locking constraints. That is, we use spinlocks as markers for regions of preemptibility. Of course, on UP (uni-processing) they aren't actually spinlocks, just markers.

The improvement to response is clear: a high priority task can run as soon as it needs to. This is a requisite of real-time computing, where you need your RT task to run the moment it becomes runnable. But the same effect applies to normal interactive tasks: as soon as an event occurs (such as the user clicking the mouse) that marks it runnable, it can run (subject to the non-preemptible regions, of course).

There are some counterarguments. The first is that the preemptible kernel lowers throughput since it introduces complexity. Testing has showed, however, that it improves throughput in nearly all situations. My hypothesis is that the same quicker response to events that helps interactivity helps throughput. When I/O data becomes available and a task can be removed from a wait queue and continue doing I/O, the preemptible kernel allows it to happen immediately -- as soon as the interrupt that set need_resched returns, in fact. This means better multitasking.

There are other issues, too. We have to take care of per-CPU variables, now. In an SMP kernel, per-CPU variables are "implicitly locked" -- they don't have explicit locks but since they are unique to each CPU, a task on another CPU can't touch them. Preemption makes it an issue since a preempted task can trample on the variables without locks.

Overall I think the issues can be addressed and we can have a preemptible kernel as a proper solution to latency in the kernel.

Note: The following question was added following initial publication of this interview, in order to clarify how automatic (or not) the effects of the preemptible kernel patch are on a typical user's system . . .

RL: From your brief explanation of how the preemptible kernel does its business, I get the impression that nothing new needs be done on a user's system other than installing the modified kernel in order to gain benefits. Specifically, if I were to install the patched kernel on my desktop system, would I immediately begin noticing performance improvements during my work (and play) on the system, such as doing work during downloads, or burning CDs while listening to streaming audio or watching streaming video? Or, do I need to have apps which can take advantage of the kernel's preemption capability? Would it be necessary to alter the priority of the apps I run, or make any other adjustments?

Love: You don't have to do anything. The preemptible kernel patch requires no change in user applications, environment, or API. The change (from switching to a preemptible kernel) should be immediate. Your normal programs and your normal benchmarks will show latency improvements. Renicing your programs would help, just because they would be able to reap the benefits of preemption more readily (anything they need to do could cause a preemption).

This is why I see elegance in this solution; Linux can be made more RT without abandoning itself.

RL: Please describe yourself, what you do, and what your interests are. Also, what is your background relative to Linux, and your philosophy with respect to open source software?

Love: I am a student at the University of Florida studying Mathematics and Computer Science, originally from South Florida. I first used Linux six or seven years ago with an early Slackware release. I've run nothing but Linux for the past couple years.

Outside of computing I'm really into rock music, good food, and the military-industrial complex.

My philosophy with respect to open source is that of a pragmatist. While I enjoy the freedom of the code and certainly appreciate the rights gained from free software, I see the biggest benefit in merely having open code. I can see it, I can fix it, I can play with it. With enough momentum, open source creates amazing software. I am very proud of our kernel.

RL: Approximately how much time per week do you spend working on your kernel patch for Linux?

Love: My girlfriend would probably say too much. Anywhere from a couple hours a week to many hours a day.

RL: What is your vision of the future of Linux in the Embedded market? in the home and business Desktop Computer market?

Love: Linux is going far in the embedded market. I'm sure you as much as anyone can see the vast potential there. Linux offers key benefits -- particularly available source and a highly configurable system -- that should appeal to the embedded space. It is light so it embeds well. It has a simple API. A lot of userspace solutions exist and it is easy to create entirely new ones.

I certainly think Linux is an excellent desktop system (both GNOME and KDE are quite mature) but I don't see huge strides here in the coming year. Market share will certainly increase, perhaps significantly, but we will not be overtaking any evil monopolies this year.

RL: Is there anything else you'd like to communicate to the LinuxDevices.com community?

Love: The latest version of all this fun stuff is available for ftp download here, or at your favorite mirror.

RL: Thanks very much!

Related stories:

Talk back! Do you have a comment or question about this story? talkback here

(Click here for further information)

7 Advantages of D2D Backup
For decades, tape has been the backup medium of choice. But, now, disk-to-disk (D2D) backup is gaining in favor. Learn why you should make the move in this whitepaper.

4 Legal Reasons to Control Internet Access
The Internet is obviously a valuable resource for many organizations. However, many are exposed to legal liability concerns because they fail to control Internet access. Learn if you're safe in this white paper.

Rapidly Resolve J2EE Application Problems
Whether you are in the process of building J2EE applications or have J2EE applications already running in production, you must ensure that they deliver the expected ROI. Learn how in this white paper.

Load Testing 2.0 for Web 2.0
There are many unknowns in stress testing Web 2.0 applications. Find out how to test the performance of Web 2.0 in this white paper.

Build Better Games Online
For the game infrastructure providers, life is complex. Making money from games has become more complicated. Why? Find out in this white paper.

Building a Virtual Infrastructure from Servers to Storage
This white paper discusses the virtual storage solutions that reduce cost, increase storage utilization, and address the challenges of backing up and restoring Server environments.

Gaining Faster Wireless Connections with WiMAX
Welcome to what is quickly becoming the hyperconnected world where anything that would benefit from being connected to the network will be connected. Learn more in this white paper.

Is Your Desktop a Security Threat?
The new wave of sophisticated crimeware not only targets specific companies, but also targets desktops and laptops as backdoor entryways into those business’ operations and resources. Learn how to stay safe in this white paper.

Increasing SAN Reliability by 100 Percent
Storage area networks (SAN) are a strong part of storage plans. Learn how to increase your reliability and uptime by 100 percent in this case study.

Got a HOT tip? please tell us!

Click here for a profile of each sponsor:

PLATINUM SPONSORS
<a href="http://ad.doubleclick.net/jump/entzd.linuxdevices/desktop;kw=timesys;pos=top;dcopt=ist;zdid=a221772;tile=1;sz=150x30;ord=<script=JavaScript>document.write(Math.round(Math.random()10000000000));</script>?"" style="border-width:0px;height:150px;width:30px;"><img src="http://ad.doubleclick.net/ad/entzd.linuxdevices/desktop;kw=timesys;pos=top;dcopt=ist;zdid=a221772;tile=1;sz=150x30;ord=<script=JavaScript>document.write(Math.round(Math.random()10000000000));</script>?" style="border:0; height: 150; width:30;" /></a>		<a href="http://ad.doubleclick.net/jump/entzd.linuxdevices/desktop;kw=windriver;pos=top;dcopt=ist;zdid=a221772;tile=1;sz=150x30;ord=<script=JavaScript>document.write(Math.round(Math.random()10000000000));</script>?"" style="border-width:0px;height:150px;width:30px;"><img src="http://ad.doubleclick.net/ad/entzd.linuxdevices/desktop;kw=windriver;pos=top;dcopt=ist;zdid=a221772;tile=1;sz=150x30;ord=<script=JavaScript>document.write(Math.round(Math.random()10000000000));</script>?" style="border:0; height: 150; width:30;" /></a>

<a href="http://ad.doubleclick.net/jump/entzd.linuxdevices/desktop;kw=versalogic;pos=top;dcopt=ist;zdid=a221772;tile=1;sz=150x30;ord=<script=JavaScript>document.write(Math.round(Math.random()10000000000));</script>?"" style="border-width:0px;height:150px;width:30px;"><img src="http://ad.doubleclick.net/ad/entzd.linuxdevices/desktop;kw=versalogic;pos=top;dcopt=ist;zdid=a221772;tile=1;sz=150x30;ord=<script=JavaScript>document.write(Math.round(Math.random()10000000000));</script>?" style="border:0; height: 150; width:30;" /></a>		<a href="http://ad.doubleclick.net/jump/entzd.linuxdevices/desktop;kw=winsystems;pos=top;dcopt=ist;zdid=a221772;tile=1;sz=150x30;ord=<script=JavaScript>document.write(Math.round(Math.random()10000000000));</script>?"" style="border-width:0px;height:150px;width:30px;"><img src="http://ad.doubleclick.net/ad/entzd.linuxdevices/desktop;kw=winsystems;pos=top;dcopt=ist;zdid=a221772;tile=1;sz=150x30;ord=<script=JavaScript>document.write(Math.round(Math.random()10000000000));</script>?" style="border:0; height: 150; width:30;" /></a>

<a href="http://ad.doubleclick.net/jump/entzd.linuxdevices/desktop;kw=virtuallogix;pos=top;dcopt=ist;zdid=a221772;tile=1;sz=150x30;ord=<script=JavaScript>document.write(Math.round(Math.random()10000000000));</script>?"" style="border-width:0px;height:150px;width:30px;"><img src="http://ad.doubleclick.net/ad/entzd.linuxdevices/desktop;kw=virtuallogix;pos=top;dcopt=ist;zdid=a221772;tile=1;sz=150x30;ord=<script=JavaScript>document.write(Math.round(Math.random()10000000000));</script>?" style="border:0; height: 150; width:30;" /></a>		<a href="http://ad.doubleclick.net/jump/entzd.linuxdevices/desktop;kw=trolltech;pos=top;dcopt=ist;zdid=a221772;tile=1;sz=150x30;ord=<script=JavaScript>document.write(Math.round(Math.random()10000000000));</script>?"" style="border-width:0px;height:150px;width:30px;"><img src="http://ad.doubleclick.net/ad/entzd.linuxdevices/desktop;kw=trolltech;pos=top;dcopt=ist;zdid=a221772;tile=1;sz=150x30;ord=<script=JavaScript>document.write(Math.round(Math.random()10000000000));</script>?" style="border:0; height: 150; width:30;" /></a>

<a href="http://ad.doubleclick.net/jump/entzd.linuxdevices/desktop;kw=montavista;pos=top;dcopt=ist;zdid=a221772;tile=1;sz=150x30;ord=<script=JavaScript>document.write(Math.round(Math.random()10000000000));</script>?"" style="border-width:0px;height:150px;width:30px;"><img src="http://ad.doubleclick.net/ad/entzd.linuxdevices/desktop;kw=montavista;pos=top;dcopt=ist;zdid=a221772;tile=1;sz=150x30;ord=<script=JavaScript>document.write(Math.round(Math.random()10000000000));</script>?" style="border:0; height: 150; width:30;" /></a>		<a href="http://ad.doubleclick.net/jump/entzd.linuxdevices/desktop;kw=calaosystems;pos=top;dcopt=ist;zdid=a221772;tile=1;sz=150x30;ord=<script=JavaScript>document.write(Math.round(Math.random()10000000000));</script>?"" style="border-width:0px;height:150px;width:30px;"><img src="http://ad.doubleclick.net/ad/entzd.linuxdevices/desktop;kw=calaosystems;pos=top;dcopt=ist;zdid=a221772;tile=1;sz=150x30;ord=<script=JavaScript>document.write(Math.round(Math.random()10000000000));</script>?" style="border:0; height: 150; width:30;" /></a>

<a href="http://ad.doubleclick.net/jump/entzd.linuxdevices/desktop;kw=technologicsystems;pos=top;dcopt=ist;zdid=a221772;tile=1;sz=150x30;ord=<script=JavaScript>document.write(Math.round(Math.random()10000000000));</script>?"" style="border-width:0px;height:150px;width:30px;"><img src="http://ad.doubleclick.net/ad/entzd.linuxdevices/desktop;kw=technologicsystems;pos=top;dcopt=ist;zdid=a221772;tile=1;sz=150x30;ord=<script=JavaScript>document.write(Math.round(Math.random()10000000000));</script>?" style="border:0; height: 150; width:30;" /></a>		<a href="http://ad.doubleclick.net/jump/entzd.linuxdevices/desktop;kw=technicalsolutions;pos=top;dcopt=ist;zdid=a221772;tile=1;sz=150x30;ord=<script=JavaScript>document.write(Math.round(Math.random()10000000000));</script>?"" style="border-width:0px;height:150px;width:30px;"><img src="http://ad.doubleclick.net/ad/entzd.linuxdevices/desktop;kw=technicalsolutions;pos=top;dcopt=ist;zdid=a221772;tile=1;sz=150x30;ord=<script=JavaScript>document.write(Math.round(Math.random()10000000000));</script>?" style="border:0; height: 150; width:30;" /></a>

GOLD SPONSORS
<a href="http://ad.doubleclick.net/jump/entzd.linuxdevices/desktop;kw=ultimate;pos=top;dcopt=ist;zdid=a221772;tile=1;sz=150x30;ord=<script=JavaScript>document.write(Math.round(Math.random()10000000000));</script>?"" style="border-width:0px;height:150px;width:30px;"><img src="http://ad.doubleclick.net/ad/entzd.linuxdevices/desktop;kw=ultimate;pos=top;dcopt=ist;zdid=a221772;tile=1;sz=150x30;ord=<script=JavaScript>document.write(Math.round(Math.random()10000000000));</script>?" style="border:0; height: 150; width:30;" /></a>		(Become a sponsor)

ADVERTISEMENT
(Advertise here)

Check out the latest Linux powered...

mobile phones!

other cool
gadgets

HOWTOs: from DevShed & IBM DeveloperWorks:

BREAKING NEWS

• Linux device monitors networks
• Linux gets security black eye
• Debian adding low-power NAS devices
• Microsoft frees poor children from Linux struggles
• Open source awards open for nominations
• Atom-based ECX board runs Linux
• Tiny boards gain Linux cross-tools support
• Cross-platform tools vendor announces awards, earnings
• Little thin client runs Linux
• $7 Soc runs Linux
• Linux wins big in financial trading
• Linux-based remote access equipment adds monitoring
• Dual-core ARM SoC clocks to 1.2GHz
• Verizon Wireless, seven others join Linux phone org
• Android Developer Challenge announces first-round winners

Linux-Watch headlines:
• Verizon chooses Linux "platform of choice"
• Hats off to Fedora 9
• Running a small business on desktop Linux
• Sun launches OpenSolaris
• Via tiptoes toward openness
• Linux certification comes to Italy
• Installing Ubuntu Hardy Heron as a web hosting server
• Black Duck Buys Koders
• Open source conference co-locates with Ubuntu show
• Ubuntu 8.04 ready to challenge Windows

Also visit our sister site:

Sign up for LinuxDevices.com's...

news feed

Use of this site is governed by our Terms of Service and Privacy Policy. Except where otherwise specified, the contents of this site are copyright © 1999-2008 Ziff Davis Enterprise Holdings Inc. All Rights Reserved. Reproduction in whole or in part in any form or medium without express written permission of Ziff Davis Enterprise is prohibited. Linux is a registered trademark of Linus Torvalds. All other marks are the property of their respective owners.