Gap Analysis: technique to work with complex systems
Overview
I have been doing this from time to time in my numerous engineering projects. Hence I thought of formalizing it to seek feedback from others to see whether there are other alternative methodologies or any further fine tuning to this approach.
Problem at Hand
Let’s say you are building a very complex system and it is not working as expected. May be it is too slow, takes more space, hogs more network bandwidth etc. How do you go about troubleshooting, narrowing down, fixing and ultimately verifying the fix so that your system starts working as expected.
Methodology
What you do in that case is determine a parameter that you want to measure, say the time it takes to run, T. You then determine, based on your understanding of the system, other measurable parameters of the system that you think impact this time, T, say A, B, C. You formulate the correlation, say
F(A,B,C) = T
Then you conduct a number of experiments with various inputs measuring values of all these parameters listing them as
Now you augment this table with your expected value as
Then you measure and monitor the gap between observed and expected, as per observed values of correlating parameters, values of your pivot parameter, T.
Possible Outcomes
You may come up with some additional parameter, say D, that you missed earlier that impacts T. You may realize that your system has issues handling certain inputs. You may come up with some other function, f’, which represents the correlation more precisely etc.
Conclusion
I think this Gap Analysis is very widely applicable working with numerous systems. Of course identifying pivot parameter T is the key and comes with domain expertise that knows how the system works. Identifying correlating parameters A, B, C and the correlation function, f, comes next. Determining the inputs, their distribution can be important in some cases. The experiments might have to be conducted in a controlled environment and values of various parameters carefully measured. In general, it is a very useful methodology in my experience. Any further fine tuning and / or alternatives you would like to suggest?
