Relational Systems set back DP by at least 10 yrs.

This is a complete reproduction of a letter to the editor of Commmunications of the ACM, published in April 1992 issue. This letter was written in response to an article by Ullman et al. that appeared in the Oct 91 issue of CACM (which was devoted to "Next Generation Databases") which said, "Relational Databases was one of the three major achievements of the past two decades". ------------Begin letter ----------------------------------------- Copyright ACM 1992. Henry G. Baker, Nimble Computer Corp, 16231 Meadow Ridge Way, Encino,CA 91436. Relational databases: I had great difficulty in controlling my mirth while I read the self-congratulatory article, "Database Systems: Achievements and Opportunities" (Oct 1991, p110), because its authors consider _relational databases_ to be one of the greatest achievements of the past decades. As a designer of commercial manufacturing applications on IBM mainframes in the late 1960s and early 1970s, I can categorically state that relational databases _set the data processing industry back at least ten years_ and wasted many of the billions of dollars that were spent on data processing. With the recent arrival of object-oriented databases, the industry may finally achieve some of the promises which were made 20 years ago about the capabilities of computers to automate and improve organizations. Biological systems follow the rule of "ontogeny recapitulates phylogeny", which states that every organism goes through a developmental history which mirrors the evolutionary development of the species itself. Data processing systems seem to have followed the same rule in perpetuating the Procrustean bed of the "unit record". Virtually all commercial applications in the 1960's were based on _files_ of fixed-length _records_ of multiple _fields_, which were _selected_ and _merged_. Codd's relational theory dressed up these concepts with the trappings of mathematics (wow, we lowly Cobol programmers are now _mathematicians_!) by calling file _relations_, records _rows_, fields _domains_, and merges _joins_. To a close approximation, established data processing _practice_ became database _theory_ by simply renaming all of the concepts. Because "algebraic relation theory" was much more respectable than "data processing", database theoreticians could now get tenure at respectable schools whose names did not sound like the "Control Data Institute". Unfortunately, relational databases performed a task that did not need doing: e.g., these databases were orders of magnitude slower than the "flat files" they replaced, and they could not begin to handle the requirements of real-time transaction systems. In mathematical parlance, they made trivial problems _obviously_ trivial, but did nothing to solve the really hard data processing problems. in fact, the advent of relational databases made the hard problems harder, because the application engineer now had to convince his nontechnical management that the relational database had no clothes. Why were relational databases such a Procrustean bed? Because organizations, budgets, products, etc., are _hierarchical_; hierarchies requires transitive closures for their "explosions"; and transitive closures cannot be expressed within the classical Codd model using only a finite number of joins (I wrote a paper in 1971 discussing this problem). Perhaps this sounds like 20-20 hindsight, but most manufacturing databases of the late 1960's were of the "Bill of Materials" type, which today would be characterized as "object-oriented ". Parts "explosions" and budgets "explosions" were the norm, and these databases could easily handle the complexity of large amounts of CAD-equivalent data. These databases could also respond quickly to "real-time" requests for information, because the data was readily accessible through pointers and hash tables --without performing "joins". I shudder to think about the large number of man-years that were devoted during the 1970s and 1980s to "optimizing" relational databases to the point where they could remotely compete in the marketplace. It is also a tribute to the power of the universities, that by teaching only relational databases, they could convince an entire generation of computer scientists that relational databases such as flat files and Bill of Materials. Computing history will consider the past 20 years as a kind of Dark Ages of commercial data processing in which the religious zealots of the Church of Relationalism managed to hold back progress until a Renaissance rediscovered the Greece and Rome of pointer-based databases. Database research has produced a number of good results, but the relational database is not one of them. Henry G. Baker -----------------------------------------------------------------------