Article II for Good Architecture: Isolation of Logic By Application Area (Where I Break with Many COTS Vendors)
Shortcuts Lead to Long Delays
Most of us have been presented with the challenge of rapidly developing a product to beat the competition to market, support a major event or simply please a stakeholder. In these situations it is easy to “give in” and take a few shortcut to get to market sooner. Some of the most common shortcuts I have seen involving placing logic in whichever place has the fastest upfront coding time. The damage these cause is significant
Shortcut Number 1: Placing Too Much Application Logic in the User Interface
Rational: UI development is rapid and often less technologically complex than back end server development (it can done by less senior engineers and visually checked very rapidly). As a result, many team place too much business logic in the UI (either in work flows or JavaScript).
While this does speed initial time to market, it creates many problems.
Problems that Emerge Immediately: Loading pages with JavaScript makes them slow (a bad user experience). It also makes them error prone, as each browser type can render JavaScript differently and some people turn it off.
Problems that Emerge in Month 2: Business rule-based application logic requirements change often. Now, when you make a change in business rules or application logic you need to change your UI. This creates usability and training problems. It also is expensive, as more testing is needed to certify UI vs. application rule changes. In addition, as time progresses more and more of your application logic spread out across your back end and UI (making your application harder or costlier to maintain).
Problems That Emerge in Quarter 2: Once you want to extend your application to a new client (call center client, mobile client, etc.) you either have to move all your application logic to the backend or re-produce it in your second (and third) UI clients. At this point, your “shortcut” is now a “long one.”
Shortcut Number 2: Hard-Coding Business Logic
Rational: Making rules and settings configurable (and enabling end users to administer these without changing your application) takes time and effort. Building more complex rule engines is even harder. Simply hard-coding your logic is much faster
While this does speed initial time to market, it too creates many problems.
Problems that Emerge Immediately: Unless you get the logic wrong, you rarely see immediate problems. (See the next paragraph for situations when you get the logic wrong.)
Problems that Emerge in Month 2: Business rule-based application logic requirements change often. Now, when you make a change in business rules you need to go deep inside the code to make changes. This requires testing of the change and regression testing of the whole application. This is risky, time-consuming and expensive.
Problems That Emerge in Quarter 2: Eventually you will need to make enough changes that you turn your application in “spaghetti code.” If you keep your original development team, you can manage this. If you switch team members, you will lose much time waiting for the new team to “hunt and peck” through the code base to figure out how to change things. Eventually (usually within 6-9 months) the team will tell you they want to re-build the application. Your “shortcut” has now led to creation of a whole new project.
Shortcut Number 3: Placing Too Much Application Logic in the Database
Rational: Database Management Systems (DBMSs) come with whole sets of functionality to manage transaction completion and referential integrity. In addition, SQL is a higher-level domain-specific language. As such, building logic with Stored Procedures or Dynamic SQL is easier and faster than building the equivalent in Java, C++, C# or another language.
While this does speed initial time to market, it too creates many problems.
Problems that Emerge Immediately: You add load to the database. In general, it is much more difficult to scale stateful database capacity (see this post) than it is to scale stateless application servers. If you get enough of a spike in traffic, all of your users suffer. You can solve this with a bigger box – but this means more CPUs and less reliability (just compare the MTBF for a 192-CPU server to a 2-CPU server).
Problems that Emerge in Month 2: Business rule-based application logic requirements change often. Now, when you make a change in business rules you are modifying how you manage database information. This requires changes to how you manage your data model (changes that pose significant risk and require high levels of regressing testing). In addition, as time progresses you have more and more of your application logic spread out across your application and database (making your application harder or costlier to maintain).
Problems That Emerge in Quarter 2: Eventually you will need to make enough changes to your logic that you will break your data model (not only adding columns but now creating new entities and relationships). Changing the data model of an operational, large-scale database is akin to changing the engine of an in-flight aircraft. It can be done, but it is risky, costly and requires someone with real-world experience doing this. At this point, your “shortcut” is now a “dead end.”
Depicting this in the Constitution of Good Architecture
Introducing the concept of Isolation of Logic not only circumvents these problems, it also enables you to create world-class applications that can “turn on a dime” in response to business changes. This is that High Repeatability / High Configurability Goal that all CEOs ask their CIOs and CTOs to achieve.
Isolation of Logic by Type was second article I defined for good architecture. Here us how I wrote it in the language of software architecture:
Article. II. Isolation of Logic by Type
Section. 1.
Applications shall only include presentation and navigation logic at the UI layer.
Section. 2.
Applications shall only employ data management systems for to explicit purpose of searching, accessing and updating data.
Section. 3.
Applications shall modularly separate business logic from application logic in order to enable changes in business policy without changes in executable code.
Section. 4.
Applications shall modularly separate business logic from presentation logic to enable rapid, low cost internationalization, localization, and branding.
Section. 5.
Modules shall soft-code all business logic, operational configuration and initialization settings. Soft-coding externalizes all configurable variables to a repository capable of supporting dynamic variable change and incorporation by all referencing modules without the need to “bounce” instances of the application in production or modify, recompile, rebuild, and re-release the underlying application source code base
You will notice that, I did not use specific technologies. This is because these principles hold for regardless of where we are in the technology curve (e.g., ten years ago, we used thick clients to enable back end servers to validate user input without requiring a slow HTTP Post, now we can use AJAX to do this “on the fly.”) In addition, a whole industry of rule and workflow engines developed to enable easy isolation of business logic from transaction and state management.
Why Isn’t Isolation of Logic by Type Used Universally?
The principles outlined above seem like “common sense.” So why isn’t everyone using them? I have seen three top reasons over and over:
Reason #1: Lack of Experience
The best way you can learn not to put logic in the wrong place(s) (or to hard-code it) is to get “burned” by doing so. The best way you can learn to make soft coding of logic something that does not need software engineers is to be in an environment of rapid change. I was blessed to be at AOL where, 1) I needed to learn this to survive, and 2) I had access to people who could show me how to do it.
Since learning and repeating these principles at multiple companies I have found that once you expose a team to this approach, they never go back.
Reason #2: “Ready-Fire-Aim” Mentality
I have never met a stakeholder that wants to wait longer for a system or application. Almost all of the time, stakeholders challenge teams to cut the time or cost on projects. This leads to short-term shortcuts.
However, no stakeholder – business or technical – wants to create technology that is inflexible or hard to maintain and adapt to evolving business needs. If you start from this premise, and collaborate both on what business needs are critical for launch and how the business will evolve over time, you can almost always come to an agreement that balances speed to market with creation of flexible, adaptable technology. (This is not a theoretical platitude; it is something that I have experienced over and over as both a business and technology stakeholder.)
Reason #3: License-based COTS Vendors Don’t Want You to Do This
Many of the commercial-of-the-shelf (COTS) vendors do not follow these principles. The most common violation is putting too much logic in the database. Vendors do this because it lets them adjust their applications more easily to many customers. This is fine during implementation (when there is no load on the system) but can become highly problematic later (when you are adding customizations, adapting to an evolving business model or performing an application upgrade).
The worst case I encountered was at a company that was the second or third largest user in the world (depending on whether you counted transaction volume or data volume) of a CRM package that created a whole business logic programming language around SQL. (What was even worse was the fact that this programming language was evoked from UI—forms, essentially splitting all business logic across the UI and the database—the worst possible scenario).
This company needed to upgrade their CRM package (their current version was no longer supported by the vendor). Unfortunately, the tight coupling of logic to the database would have required a multi-day application and database outage (effectively shutting down their business). As a result, they moved to different vendor (one that split business logic from the UI, database or even transaction processing, via use of rule and workflow engines). This move not only circumvented the problem, it also gave them a better CRM platform.
(Call me if you want the details on this problem and the vendors they used.)
How Can You Test Effect Isolation of Logic?
Take a look at your portfolio of projects over the past 12 months. If you see many tactical projects that required software engineers to make changes in UI, workflow, business rules, etc., you likely have insufficient isolation of logic. If these projects took more than 16-24 hours of work on average you definitely have a problem (if it took less, you may simply want to look at improved process and administration tools to enable business analysts to make these changes).
