|
Articles Index
Real-time computing is often associated with high speed, but this
is only one part of the picture. At its core, real-time computing is
about predictability -- the knowledge that the system will
always perform within the required time frame. The deadlines involved
need not be very short -- though they sometimes are -- and the
consequences of missing a deadline may not be dire -- though they
sometimes are. The key to whether an application is a real-time one
has to do with whether its requirements include temporal constraints.
The developers of the Real-Time Specification for Java (RTSJ), JSR
1, use this definition of real-time computing:
The programming environment must provide abstractions
necessary to allow developers to correctly reason about the temporal
behavior of application logic. It is not necessarily fast, small, or
exclusively for industrial control; it is all about the
predictability of the execution of application logic with respect to
time.
Real-time applications are partitioned into a set of tasks, some
of which have deadlines. The goal is for all tasks to be scheduled in
such a way as to complete before their deadline, if any.
Definitions: Hard-Real-Time and Soft-Real-Time
Real-time systems may be classified differently depending on the
system's requirements. A hard-real-time system is one in which
the system must meet all deadlines without fail. Typically, such
systems also have short latencies, the time between when a
triggering event occurs and when the response must begin or complete,
often measured in microseconds or milliseconds.
Many hard-real-time systems are further classified as
safety-critical systems. These systems are used to protect
humans from injury or danger. Safety-critical systems must go through
exhaustive testing and a line-by-line code review before
certification. Note that safety-critical Java technology is being
studied through JSR
302 and is not addressed by the RTSJ at the time of this writing.
A soft-real-time system, on the other hand, is one that
will still function correctly, according to its specification, if the
system occasionally misses deadlines. For example, a cellular
telephone switching system is a soft-real-time system: Customers
expect all calls to be completed, but an occasional deadline miss may
cause an acceptable failure, such as audio dropouts or delayed call
setup. Soft-real-time systems typically specify what percentage of
deadlines can be missed or how often this is acceptable.
Unpredictability in a Java Technology-Based Application
A number of factors may render the timing of execution
unpredictable and therefore may cause a standard Java task to miss
its deadline. Here are the most common.
- Operating-system scheduling. In Java technology, threads
are created by the JVM* but are ultimately
scheduled by the operating-system scheduler. In order for the JVM to
provide guarantees of temporal latency, that is, the time delay in
reaction after an action, the operating system must provide
scheduling-latency guarantees as well. Hence, the operating system
must offer capabilities such as a high-resolution timer,
program-defined low-level interrupts, and a robust priority-based
scheduler.
- Priority inversion. One hazard in an application in which
threads can have different priorities is priority inversion.
If a lower-priority thread shares a resource with a higher-priority
thread, and if that resource is guarded by a lock, the lower-priority
thread may be holding the lock at the moment when the higher-priority
thread needs it. In this case, the higher-priority thread is unable
to proceed until the lower-priority thread has completed its work --
and this can cause the higher-priority thread to miss its deadline.
In addition, if some other task with a medium priority that does not
depend on the shared resource attempts to run in the interim, it will
take precedence over both the low- and the high-priority tasks. The
effect of priority inversion is to "downgrade" the priority
of the higher-priority thread to that of the lower-priority thread.
- Class loading, initialization, and compilation. The Java
language specification requires classes to be initialized lazily --
when an application first uses them. Such instantiations might
execute user code, creating jitter, a variance in latency, the
first time that a class is used. In addition, the specification
allows classes to be lazily loaded. Because loading of classes may
require going to disk or across the network to find the class
definition, referencing a previously unreferenced class can cause an
unexpected -- and potentially huge -- delay. In addition, the JVM has
some latitude to decide when, if ever, to translate a class from byte
code into native code. Typically, a method is compiled only when it
is executed frequently enough to warrant the cost of compilation.
- Garbage collection. The primary source of unpredictability
in Java applications is garbage collection (GC). The GC algorithms
that standard JVMs use all involve a stop-the-world pause, in
which the application threads are stopped so that the garbage
collector can run without interference. Applications with hard
response-time requirements cannot tolerate long GC pauses. Despite a
large amount of work in recent years on reducing GC pauses, a
so-called low-pause collector is still not enough to guarantee
predictability and still may require significant tuning and testing.
- The application. Another major source of unpredictability
is the application itself, including any libraries that it uses. Most
applications consist of multiple computational activities that
compete equally for CPU resources. Java applications typically do not
use thread priorities, partly because the JVM offers such weak
guarantees for thread priorities. In a properly provisioned
application, enough CPU cycles should be available to go around, but
completing a task may sometimes take longer than expected -- which
may cause other tasks to have to wait for CPU resources.
Most developers treat this problem iteratively: They look for the
most serious bottleneck, improve it, then find the next bottleneck
and improve it, repeating these steps until the application reaches
acceptable performance. However, because this development process is
itself unpredictable, it can often affect delivery schedules or
require significant additional testing.
- Other activities in the system. Other high-priority
activities that occur in the system -- such as hardware interrupts,
other real-time applications, and so forth -- can cause jitter in the
application, thereby affecting determinism.
The Real-Time Specification for Java (RTSJ)
The Real-Time Specification for Java (RTSJ), or JSR
1, specifies how Java systems should behave in a real-time
context and was developed over several years by experts from both the
Java and real-time domains.
The RTSJ is designed to seamlessly extend any Java family --
whether the Java Platform, Standard Edition (Java SE); Java Platform,
Micro Edition (Java ME); or Java Platform, Enterprise Edition (Java
EE) -- and has the requirement that any implementation pass both the
JSR 1 technology compatibility kit (TCK) and the TCK of the platform
-- Java SE, Java ME, or Java EE -- on which it is based.
The RTSJ introduces several new features to support real-time
operations. These features include new thread types, new
memory-management models, and other newly introduced frameworks.
Let's start with how the RTSJ views tasks and time.
Tasks and Deadlines
The RTSJ models the real-time part of an application as a set of
tasks, each with an optional deadline. The deadline of a task
is when the task must be completed. Real-time tasks fall into several
types, based on how well the developer can predict their frequency
and timing:
- Periodic: tasks that run on a fixed schedule, such as reading a sensor every five milliseconds
- Sporadic: tasks that do not run on a fixed schedule but that have a maximum frequency
- Aperiodic: tasks whose frequency and timing cannot be predicted
The RTSJ uses this task-type information in several ways to ensure
that critical tasks do not miss their deadlines. First, the RTSJ
allows you to associate with each task a deadline miss handler.
If a task does not complete before its deadline, this handler is
called.
Deadline-miss information can be used in deployment to take
corrective action or to report performance or behavioral information
to the user or to a management application. By comparison, in
non-real-time applications, failures may not become apparent until
secondary or tertiary side effects arise -- such as request timeouts
or memory depletion -- at which point it may be too late to recover
gracefully.
Deadline-miss handling is deferred to a deadline miss handler, like this:
ReleaseParameters.setDeadlineMissHandler(
AsyncEventHandler handler);
|
Or if there is no handler, it can be performed by the thread itself:
if (waitForNextPeriod() == false) {
handle_deadline_miss();
}
|
Thread Priorities
In a true real-time environment, thread priorities are extremely
important. No system can guarantee that all tasks will complete on
time. However, a real-time system can guarantee that if some tasks
are going to miss their deadlines, the lower-priority tasks are
victimized first.
The RTSJ defines at least 28 levels of priority and requires their
strict enforcement. However, as this article mentioned earlier, RTSJ
implementations rely on a real-time operating system to support
multiple priorities, as well as on the ability of higher-priority
threads to preempt lower-priority ones.
The problem of priority inversion can undermine the effectiveness
of a priority facility. Accordingly, the RTSJ requires priority
inheritance in its scheduling. Priority inheritance avoids
priority inversion by boosting the priority of a thread that is
holding a lock to that of the highest-priority thread waiting for
that lock.
This prevents a higher-priority thread from being starved because
a lower-priority thread has the lock that it needs but cannot get
adequate CPU cycles to finish its work and release the lock. This
feature also prevents a medium-priority task that does not depend on
the shared resource from preempting the higher-priority task.
In addition, the RTSJ is designed to allow both non-real-time and
real-time activities to coexist within a single Java application. The
degree of temporal guarantees provided to an activity depends on the
type of thread in which the activity is executing: java.lang.Thread
or javax.realtime.RealtimeThread thread types.
- Standard
java.lang.Thread (JLT) threads are
supported for non-real-time activities. JLT threads can use the 10
priority levels specified by the Thread class, but these are not
suitable for real-time activities because they provide no guarantees
of temporal execution.
- The RTSJ also defines the
javax.realtime.RealtimeThread
(RTT) thread type. RTTs can take advantage of the stronger thread
priority support that the RTSJ offers, and they are scheduled on
a run-to-block basis rather than a time-slicing basis. That is, the
scheduler will preempt an RTT if another RTT of higher priority
becomes available for execution.
One of the problems with automatic memory management in standard
virtual machines (VMs) is that one activity may have to "pay
for" the memory-management costs of another activity. Consider
an application with two threads: a high-priority thread (H) that does
a small amount of allocation, and a low-priority thread (L) that does
a great deal of allocation.
If H is unlucky enough to run at a time when L has consumed almost
all the available memory in the heap, the garbage collector may kick
in and run for a long time when H goes to allocate a small object.
Now, H is paying -- in the form of an incommensurately long delay --
for L's enormous memory consumption.
The RTSJ provides a subclass of RTT called NoHeapRealtimeThread
(NHRT). Instances of this subclass are protected from GC-induced
jitter. The NHRT class is intended for hard-real-time activities.
To maximize predictability, NHRTs are allowed neither to use the
garbage-collected heap nor to manipulate references to the heap.
Otherwise, the thread would be subject to GC pauses, and this could
cause the task to miss its deadline. Instead, NHRTs can use the
scoped memory and immortal memory features to allocate
memory on a more predictable basis.
The following two NHRT examples demonstrate this.
NHRT -- Example 1
public class PeriodicThread {
static NoHeapRealtimeThread nhrt;
static {
int prio = PriorityScheduler.instance().getMaxPriority();
RelativeTime period = new RelativeTime(20,0);
nhrt = new NoHeapRealtimeThread(
new PriorityParameters(prio),
new PeriodicParameters(
period),
ImmortalMemory.instance()) {
public void run() {
. . . . .
};
};
}
. . . . .
}
|
NHRT -- Example 2
public class MyNHRT extends NoHeapRealtimeThread {
public MyNHRT() {
super(new PriorityParameters(5), ImmortalMemory.instance());
setReleaseParameters(
new PeriodicParameters(
new RelativeTime(0, 0),
new RelativeTime(200, 0),
new RelativeTime(50, 0), null, null, null));
}
}
|
Memory Areas
The RTSJ provides for several means of allocating objects,
depending on the nature of the task doing the allocation. Objects can
be allocated from a specific memory area, and different memory
areas have different GC characteristics and allocation limits.
- Standard heap. Just like standard VMs, real-time VMs
maintain a garbage-collected heap, which can be used by both
standard (JLT) and RTT threads. Allocating from the standard heap
may subject a thread to GC pauses. Several different GC technologies
can balance throughput, scalability, determinism, and memory size.
NHRTs cannot use the standard heap in order to be protected from
this balancing, which is a source of unpredictability.
- Immortal memory. Immortal memory is a
non-garbage-collected area of memory. Once an object is allocated
from immortal memory, the memory used by that object will never, in
principle, be reclaimed. The primary use for immortal memory is so
that activities can avoid dynamic allocation by statically
allocating all the memory they need ahead of time, and managing it
themselves.
Managing immortal memory requires greater care than managing
memory allocated from the standard heap, because if immortal objects
are leaked by the application, they will not normally be reclaimed.
In this case, there is no easy way to return memory to the immortal
memory area. Note: If you are familiar with C/C++, you will probably
recognize immortal memory as being similar to malloc()
and free().
- Scoped memory. The RTSJ provides a third mechanism for
allocation called scoped memory, which is available only to
RTT and NHRT threads. Scoped-memory areas are intended for objects
with a known lifetime, such as temporary objects created during the
processing of a task. Like immortal memory, scoped-memory areas are
uncollected, but the difference is that the entire scoped-memory
area can be reclaimed at the end of its lifetime, such as when the
task finishes.
Scoped-memory areas also provide a form of memory budgeting. The
maximum size of the scoped area is specified when it is created, and
if you attempt to exceed the maximum, an OutOfMemoryError
is thrown. This ensures that a rogue task does not consume all the
memory and thereby starve other -- perhaps higher-priority -- tasks
of memory.
|
Why Scoped Memory?
In Java programs, it is common to allocate temporary objects in
the course of a computation. For example, concatenating two strings
generates a StringBuffer object that is thrown away as
soon as the result string is available.
The standard programming style for Java programs encourages
producing a certain amount of garbage as a side effect of nearly any
computation, and many libraries do just that. Because not being able
to use library code from hard-real-time tasks would be unacceptably
limiting, there needs to be a mechanism to make the costs of
allocation and deallocation of such temporary objects predictable.
Scoped memory does just that. It provides the task with a means to
say "I'm going to use this much temporary memory during the
course of this task, and when the task is over, reclaim it all."
Because the memory is preallocated when the task starts, the
allocations will not fail. Because all the memory in the scoped area
is freed when the task completes, deallocation is fast and
predictable. Therefore, as long as real-time tasks can accurately
estimate how much memory they'll need to accomplish their work,
scoped memory can eliminate the unpredictability of temporary object
allocation.
However, one downside of scoped memory is that, because all the
objects in it will disappear after the task completes, you cannot
store a reference to an object in a scoped-memory area into the
immortal or heap areas. If you need to create an object with a longer
lifetime, you will have to create it in a different scope. You can
create a hierarchy of scopes, with each child having a shorter
lifetime than its parent.
In a procedure similar to the checks for null pointers or array
bounds that any JVM performs, the real-time JVM dynamically performs
assignment checks to guarantee that an object is never referenced by
an object with a longer lifetime.
|
For each thread, there is always an active-memory area, called the
current allocation context. All objects allocated by a thread
are allocated from this area. The current allocation context changes
when you execute a block of code with a specific memory area.
The memory area API contains an enter (Runnable)
method that causes the specified task to be executed using that area
as the current allocation context. Therefore, even if your code uses
third-party libraries that allocate memory, you can still use that
code with scoped-memory areas -- and all the temporary objects
allocated by that code will go away when the current allocation
context is finalized.
Advanced Communication Between Threads
One of the advantages of the RTSJ is that it allows real-time and
non-real-time activities to coexist within a single VM. However,
communication between threads involves memory, and this poses a
challenge. The obvious mechanism for communication between threads is
a queue, where one thread is putting data onto the queue and the
other thread is removing data from the queue.
Imagine that an RTT is putting data onto a queue every 10
milliseconds, and a non-RTT is consuming the data from the queue.
What happens if the non-RTT does not get enough CPU time to drain the
queue? The queue will grow, and you have a choice of three bad
options:
- The queue could be allowed to grow without bound, potentially running out of memory.
- Data from the queue must be discarded.
- The RTT will have to block.
The first option is impractical, and the last option is
unacceptable. The predictability of real-time activities should not
be affected by the behavior of lower-priority activities. The RTSJ
supports several types of non-blocking queues for communicating
between threads. When they are used between real-time and non-RTTs,
there are some restrictions on the memory area in which elements must
reside.
Part 2 of this series will examine the issue of real-time garbage collection and the benefits of the Sun Java Real-Time System.
_______
* As used on this web site, the terms "Java
Virtual Machine" and "JVM" mean a virtual machine for
the Java platform.
Acknowledgments
The editor of this article wishes to thank David Holmes and Antonia Lewis for their work on the white paper on which this article is based. And thanks to Carlos Lucasius for his valuable technical review comments.
For More Information
Real-Time Specification for Java (RTSJ), JSR 1
Sun Java Real-Time System (Java RTS): Frequently Asked Questions (FAQ)
Training: Developing Real-Time Applications for the Java Platform (DTJ-4103)
JavaOne 2007
and 2008
Hands-On Labs: How to Build Real-Time Solutions for Real-World Devices
Brian Goetz, a senior staff engineer at Sun, has been a
professional software developer for 20 years. He is the author of
over 75 articles on software development, and his book, Java Concurrency in Practice, was
published in May 2006 by Addison-Wesley. He serves on the JCP Expert
Groups for JSRs 166, concurrency utilities; 107, caching; and 305,
annotations for safety analysis.
Robert Eckstein has worked with Java technology since its
first release. In a previous life, he has been a programmer and
editor for O'Reilly Media, Inc. He has written or edited a number of
books, including Java Swing, Java Enterprise Best
Practices, Using Samba, XML Pocket Reference, and
Webmaster in a Nutshell.
|
|
Print-friendly Version
