During a recent client meeting, we were reviewing the proposed business architecture for a new system. Decision Management will play a crucial role in the application and the rule processing had been nicely delineated in the architecture. While the rules themselves had been fairly well articulated, their “insertion” in the requirements suggested quite a bit of work remained. However, I was generally pleased with the state of the work to date.
The discussion then moved to some pre-processing that would dictate both the business events to be processed and the order in which they would be executed. Doing the latter accurately was absolutely critical to produce the correct results. As we explored this pre-processing it started to become clear to me that a rather elegant solution could be crafted using rules. When I recommended this alternative, the looks I received suggested I had grown horns and a tail during the course of the meeting! It was only upon further reflection that I realized these people – and probably most others – had not been around during the early days of Artificial Intelligence when we talked about Expert Systems instead of Decision Management.
My first exposure to these technologies came in the mid-80s and I was fascinated by the concept of Artificial Intelligence (AI). After having spent several summers during college writing Basic programs to do inventory and basic accounting, I knew I wanted to find something else. AI was it. Many of the areas we researched back then have become commonplace today – computer vision, language understanding and generation, robotics, and expert systems. With respect to expert systems, there was no concept of representing corporate rules or business logic – it was essentially another programming technique to tackle some interesting problems. Relying on the RETE[1] algorithm for fast, efficient pattern matching, we had a wonderful approach to sift through large amounts of data and apply the rules triggered by matching data. And the idea of an Expert System was the first concept to come from this – systems that would emulate the decision-making of a human.
Initially these systems relied on sophisticated (and expensive) hardware and software. We tended to write programs in LISP and used Symbolics Lisp Machines or TI Explorers for our work. They were slow (relatively speaking) but so cool to sit down behind! There were very interesting early efforts including the legendary Mycin[2] system developed at Stanford to identify bacteria causing severe infections and recommend antibiotics. Using controlled heuristics to guide the search, the system would query the doctor seeking symptoms and data to satisfy the goal of determining the likely infection(s). While it was never used in practice, it achieved a success rate greater than that of Stanford medical school faculty in many tests. Why wasn’t it used? Partially due to the fear of liability in the case of a wrong diagnosis. Who gets sued? The doctor relying on the system? The system programmer? The other reason is one that seems foreign today – this was a standalone system and it was not integrated in the largely mainframe time-sharing world of that day.
Over time the expert system shells evolved into business rule systems and they migrated from LISP to more traditional programming languages. They also moved along with the times to become available on more accessible hardware platforms that did integrate well with other systems. The path from there started to focus on business rule management and we heard things like “agility” and “business user empowerment”. Business process management came along and we saw the light in terms of the integration with rules. We now find ourselves at another stage in the evolutionary process with decision management.
So why this trip down memory lane? I think the full potential of these platforms should again be recognized and applied when appropriate. I find it fascinating that the early definition of Expert Systems – systems that emulate the decision-making of a human – is still completely relevant today with Decision Management. A computer system that can determine whether or not a loan should be underwritten is certainly emulating human expertise. It’s the same ultimate goal but achieved in different ways. When programming expert systems we could absolutely not conceive of the possibility that a non-technical person would EVER dare try to read, understand, or modify that code (probably the same reaction you would have now if you thought about tossing some Java code over to a business analyst). But that is typically a primary goal today. In fact, the current releases of decision management platforms focus more on the tools and techniques surrounding that goal than anything else. I have no problem with that – it makes the systems more accessible and can bring tremendous agility (there’s that word!).
However, let’s not forget our roots. Rules can be a tremendously powerful programming technique from which very elegant, effective solutions to difficult problems can be created. Whether it’s data-driven (forward chaining rules) or goal-directed (backward chaining rules) we can do some pretty sophisticated things. It’s perfectly acceptable to leave the creation and modification of these rules to IT staff (pretty much every platform on the market lets you tailor the roles and responsibilities of the rule managers). We’re leaving a lot on the table when we only use a business rule management system to sequentially sift through a fairly flat rule base of business logic. Let’s not forget how we got here in the first place.
And with that I’m off to think about event generation and prioritization…just like an expert would.
[1] Forgy, Charles L. “Rete: A Fast Algorithm for the Many Pattern/Many Object Match Problem,” Artificial Intelligence, (19)1, Sept. 1982, pp. 17-37.
[2] Buchanan, B.G.; Shortliffe, E.H. (1984). Rule Based Expert Systems: The MYCIN Experiments of the Stanford Heuristic Programming Project. Reading, MA: Addison-Wesley. ISBN 978-0-201-10172-0.
