And, then there is the problem of developer enthusiasm. Wanting to please their customer, the developer may be tempted to deliver something functional without addressing the nonfunctional requirements: performance, stability, refactoring, security, extensibility, maintainability, etceteras. The more "mature" my Software Development Lifecycle (or SDLC), the more etcetras I will have. To me it seems like it would make sense to assign each of those categories to a different team, e.g., an assembly line with checkpoints, but the practice today is for one team to do it all. I guess assembly line manufacturing is for non-complex systems like modern automobiles.
But wait... there IS an aspect of today's world that is analogous to an assembly line. That is what we call continuous integration or continuous delivery. The "pipeline" consists of a series of gates or checkpoints at which we can test **every** requirement, via an automated toolchain that can also involve manual approvals for human tested things such as usabilty and architectural integrity. Products such as Jenkins, Travis-CI, and Chef Delivery are on the vanguard of this. The CI pipeline is in essence the belt of the assembly line, and teams are deploying themselves at each of the checkpoints.
This process belong to the developers and architects. What about the person who is talking to the customer? What about the customer? How exactly do we justify all the money going into the effort?
Value and ROI
ROI or return-on-investment is the value that the business (or customer) expects to derive from the effort. So the value is compared to the cost of the effort and I can immediately sense a ratio: if the value of the thing is $100,000 a year for the next 5 years, and the cost is $100,000, I can see an ROI of 500%. Now if you've been to business school, yes, you can whip out an HP calculator and do a time-value calculation that would actually put that at somewhere between 400 and 500% due to inflation, interest and other factors, but this article is not really about that level of detail. And here's why.
Typically the perceived ROI of a set of proposed improvements will vary quite a bit throughout its lifecycle. In addition to this, efforts will have intangible value, such as how well they align with strategy, to what extent they are foundational, how they mitigate risk, etc. So ROI is really just an accounting label we slap on it: the sales lift or cost savings directly attributable to the opportunity.
Strategic value can outweigh direct ROI. For example, Bill Gates may have spent $70,000 on building MS-DOS to the point where IBM was willing to accept his license terms to bundle it on the first IBM PC's. So what's the ROI on that work he did, to get the deal he had with IBM? The outcome is worth billions. But, the ROI that followed was probably more attributable to Bill's dad's law firm that wrote the license terms so cleverly, because it locked in a sustainable revenue stream that continues today.
Value capture is driven by the little details, the things that REALLY WORK for customers. Which is why, for the developer, the focus must not just be on dumping time and effort into a project, but on the details, on the perfection of the product.
When Can I Measure the Value of an Innovation?
|Prior to a Major Development Effort||Senior Management, Internal Customers ("The Business")||Before undertaking a major development, purchasing and installing a new application, etc., it is traditional to do a high level cost/benefit analysis. While business goals are certainly welcome in this context, specific functional requirements have to be left for later in order to enable agility throughout the effort.|
|Before Undertaking a Feature/Function Enhancement that Requires More than One Day of Development Team Effort (an "Epic")||Development Team, Operations Team, Internal Customers||A single developer can cost $112 – $2500 a day depending on whether you are working in Bangalore or at Facebook HQ. It behooves your managerial discpline to perform a mini ROI analysis prior to each major undertaking.|
|Any Requirement||Development Team||
It really helps when I can be very clear about the following on a very granular level:
|After a Major Development Effort||Senior Management, Operations Team, Internal Customers||After undertaking a major development, can it be shown that the expected benefits were realized? This has been very important to improve the strength of my processes. For example, when building a system, I determine metrics to help answer this question and then partner with the business to track these throughout the operational lifetime of the system.|
The Business Impact Statement
One tool that I have found helpful is to write a concise, one-sentence business impact statement for every story. (We used to write them for problem and incident reports in my operations past, so why not write them for requirements?) A business impact statement very simply explains the business functionality or value that stands to be gained if the requirement is achieved, lost if it is not. Here are some examples of these that I have written for a workstation cloud virutalization project:
For a class that supports testing the AWS Cloud:
Minimizes exposure of test credentials by temporarily creating a set of test credentials and then deactivating it immediately after it is used.
For a component that performs detailed logging across an entire application:
A logging increases the maintainability and reach of unit testing: it allows more functions to be reached by testing because the log is an output interface that is reachable by components that do not touch the UI. Exposing internal functionality via logging avoids making internal APIs less stable in order to provide access to test tools.
Logging reduces future costs in these areas:
Component reuse within the Java applications.
Qualification of new environments.
Changes to security, account permissions, etc.
Changes to configuration of user computers.
Changes to global configuration such as cloud, region, security.
Logging information can be captured from monitoring agents in response to production incidents.
Logging information can be captured from the automated deployment of any of the above.
For a Java component that takes snapshots of machine images for a point-in-time restore capability:
Amazon machine images (AMIs) can be used to save money while a system is not required, by storing a system configuration and a system's magnetic hard disks in inexpensive cloud storage.
Note that while the langauge above references some computer terminology, it also provides a clear picture of the business impact, e.g. WHY we are building it. Sure, some business eyes may glaze over, but in my experience the leaders who are making a difference are eager to consume and grasp information like this.
A Business Case for my Site-Test Project
benefit of automated testing
- saves 10%-50% developer time. developers get work done faster.
- allows bugs to be found more quickly, thus reducing business impact.
- increases the velocity of development.
- more comprehensive than manual testing; run full regression every time
- supports rapid integration testing and operations tests, useful for doing rapid deployments
benefit of this tool specifically
- creates generic platform for building tests that can test just about anything
- allows site to be tested automatically, before and after changes, and periodically
- safely manages credentials, excluding them from shared storage
- uses java, "lingua franca" of corporate IT
It's helpful to document it when alternatives have been considered.
- Use Selenium IDE. Although this tool makes it easy to record and play back user clicking and typing, it lacks the support of a full development language. This makers of this 1.0 version product recommend using the 2.0 version for this and other reasons.
- Test manually with user scripts and checklists. This is how we currently do things. It is error-prone and in many cases problems have slipped through. It does not give developers immediate feedback and is expensive to do for multiple devices. Also, it does not have the same reach as automated testing.
Consider the business value of what you are doing at every phase of the software development lifecycle. Special attention should be paid during very expensive undertakings, such as while writing software, and while acquiring and configuring new systems. Write brief business impact statements to help developers and others stay focused on the intent of the effort. When making a justification, consider benefits other than revenue lift and cost reduction: strategic value, risk reduction, velocity, and undesirability of alternative courses of action.
Let's Make Contact
Getting there should be easy and fast. Please feel to set up a call to talk about this, your bucket list, or maybe a cool product or service you're pitching.