|IBM home | Products & services | Support & downloads | My account|
|Diagnosing Java code: The future of software development|
Loyal readers: I regret to inform you that this will be the last column in the Diagnosing Java code series. I've (finally) finished my Ph.D., and I'm off to the industrial labs to help start a new research program in programming languages.
In this farewell article, let's have some fun and look into our crystal ball. We'll discuss some of the prevailing trends in the software industry and the impact we can expect these trends to have on the future of software development. We'll focus our discussion through the lens of effective software development we've used in this series over the past two and a half years. As always, let's pay particular attention to the crucial role that effective error prevention and diagnosis play in allowing us to manage our increasingly complex digital world.
There are three prevalent trends in computing to consider when examining where the industry is heading as whole. These are:
Together, these trends are reshaping the fundamental nature of software and the engineering trade-offs involved in constructing it. At a high level, the expense of software and the ubiquity of it are pushing us in the direction of greater and more pervasive abstractions, such as powerful virtual machines with well-defined semantics and security properties, allowing us to develop and maintain software more easily and across more platforms.
At the same time, the continued improvements in computing performance enable us to build these abstractions without suffering unacceptable performance degradation. I'd like to take a stab at some of the ways I believe we could construct new abstractions that would help in building the next generation of software products.
Component-based programming promises two complementary benefits, both of which will become increasingly important as the above-mentioned trends become more and more prominent. First, component-based systems allow for much greater reuse. For example, consider the myriad programs today that provide text-editing support (mail clients, word processors, IDEs, and so on). Also, consider the numerous clients that provide support for handling e-mail. Despite the number of programs offering these services, few programs handle e-mail as well as a dedicated e-mail client. Likewise, no mail program will allow text manipulation at the same level as a dedicated text editor. But why should every mail client (IDE, word processor, and so on) have to develop its own text editor? It would be so much better if there were a standard "text-editor" API that could be implemented by various third-party components. Tools such as mail clients could choose their favorite implementation of this API and plug it in. In fact, one can even imagine users assembling their environment from off-the-shelf components (such as, their favorite editor, their favorite mail client) that could be linked dynamically when an application is run.
Another benefit of a component-based model is the potential for greater testing. In the Java language as it exists today, external references to classes, such as the I/O library classes and the like, are hard-wired references that cannot be altered without recompilation. As a result, it is difficult to test parts of programs that rely on external references in isolation. For example, it is difficult to test whether a program is making proper use of the filesystem without actually allowing it to read and write from the filesystem. But reading and writing files in unit tests slows tests and requires adding more complexity (like creating a temp directory and cleaning up files after use). Ideally, we would like to separate programs from external references to the I/O libraries for the purposes of testing.
There are numerous ways in which we can formulate a component model. J2EE provides such a model at the level of objects for Web services. Eclipse provides a model for IDE components. Jiazzi provides a model in which independently compiled "units" of software can be linked to form a complete application. Each of these formulations has its use in particular contexts; we should expect to see yet more formulations in the coming years.
Ensuring that the requisite invariants are satisfied is a necessary aspect of component encapsulation. In general, a client programmer will have no way to reason how a component will behave other than what is said in the published API. Any behavior of a component that isn't included in the API is not behavior that the client programmer can rely on. If non-published behavior results in a run-time error, it will be exceedingly difficult for the programmer to diagnose the problem and repair it.
There are several research projects underway to significantly improve the sorts of invariants we can specify for a component. Some of them, such as Time Rover (see Resources for the July 2002 column), use modal logic and other logical formalisms to express deep attributes of run-time behavior.
Another approach to expressing invariants is to bolster the type system with generic types, types parameterized by other types (the topic of our the most recent series of articles in this column).
Yet another approach to adding much more powerful invariants is that of dependent types. Dependent types are types parameterized by run-time values (compare this with generic types, which are parameterized by other types).
The canonical example of dependent types is that of an array type parameterized by the size of the array. By including the size in the type, a compile-time checker can symbolically analyze the accesses to the array to ensure that no accesses are done outside the bounds of the array. Another compelling use of dependent types is that of ownership types, developed by Boyapati, Liskov, and Shrira (see Resources for a link to their original paper).
Ownership types are types of objects that are parameterized by an owner object. For example, consider an iterator over a container. It is natural to say that the iterator is owned by the container, and therefore, that the container has special access privileges to the iterator. Inner classes provide some of the same controls over access privileges, but ownership types provide a much more powerful and flexible mechanism.
Continued improvements in
Unit testing tools allow us to check that key invariants of our programs continue to hold under refactoring. Refactoring browsers provide many direct and powerful ways to modify code while preserving behavior. We are starting to see "second generation" unit testing tools that leverage static types and unit tests to mutual advantage, allowing for automatic testing of code coverage and automatic generation of tests. Refactoring browsers are adding more and more refactorings to the standard repertoire. Long range, we should look for even more sophisticated tools, such as "pattern savvy" refactoring browsers that recognize uses (or potential uses) of design patterns in a program and apply them.
We can even expect development tools to eventually leverage the unit tests to perform more aggressive refactorings. In Martin Fowler's classic text, Refactoring: Improving the Design of Existing Code, a refactoring is defined to preserve the observable behavior of a program. However, in general we are not concerned about all aspects of the observable behavior of a program; instead, we generally care about maintaining certain key aspects of the behavior. But these key aspects are exactly what the unit test suite is supposed to check!
Therefore, a refactoring browser could potentially leverage the unit test suite to determine what aspects of behavior are important. Other aspects could be aggressively modified by the refactoring browser at will, in order to simplify the code. On the flip side, the functionality of such a refactoring browser could be leveraged to check test coverage by determining the kinds of refactorings that are allowed by the unit tests and reporting them to the programmer.
Java Platform Debugger Architecture (JPDA) provides for exactly such a facility by allowing a debugger to run on a separate JVM; then we can use RMI for the remote debugging session. But, in addition to remote debugging, diagnosis can be made much more efficient by giving the programmer more control over the access points available when starting a debugger and the available views of the state of the computation during debugging.
Even with modern debuggers, programmers still have to resort to
Projects like Eclipse take this philosophy one step further and provide ways to interoperate tools to leverage the functionality of one another and provide services beyond what is possible with any of the tools in isolation. With time, we should expect this model, or others like it, to truly "eclipse" traditional all-in-one IDEs.
One long-range solution to this problem could be to embed meta-level knowledge into applications that encodes the context in which the application is run and what it is supposed to do. For example, meta-level knowledge for a word processor would include logic explaining that the program was used by humans on personal computers to produce English documents that are then read by other humans. With this knowledge encoded, an application could potentially make inferences about what a user was trying to do when something goes wrong (of course, the application would also have to determine that something was wrong in the first place).
Such meta-level knowledge is potentially a powerful mechanism for adding robustness to an application. It is also extremely dangerous, and what is most worrisome about the danger is that it is often overlooked by the strongest advocates of moving in this direction. After all, the behavior of an application that dynamically reconfigures itself can be extremely unpredictable. A user may find it quite difficult to reason how his program will behave under certain circumstances. And, the developers of the program can also find it extremely difficult to be able to be assured of its reliability. As we've seen time and again, the inability to predict and understand a program's behavior has easily predictable consequences -- buggy software.
To be clear, I really do think that adaptive software with meta-level knowledge about the context in which it's used has the potential to vastly improve the capabilities of software applications, but if we add such capabilities, we must find ways to do so that still allows us to reason effectively about our programs.
A great example of a software system that incorporates a form of meta-level knowledge and adaptability (albeit extremely limited) without sacrificing predictable behavior is the TiVo personal digital recorder (or other similar products). TiVo uses your television viewing habits to adaptively determine what shows you might like to watch, but this adaptability is stringently restricted. TiVo will always follow user directives for the shows to record regardless of any of its adaptive behavior responses. TiVo uses a very simple form of meta-level knowledge, but even as the meta-level knowledge used becomes more and more complex, we should continue to keep control over adaptive behavior. If you'll forgive a somewhat fanciful comparison from the realm of science fiction, we could follow the precedent set by Isaac Asimov. Asimovian robots were extraordinarily powerful machines, but they were controlled by absolute and inviolable fundamental laws, allowing for some degree of predictability in their behavior.
I bid you a fond
To my readers: I hope you have found some lasting value in these articles. Writing them has been an invaluable learning experience for me. Thank you for your patronage, and I wish you the best of luck in your efforts to prevent and diagnose bugs in your programs.