In an article titled “Project Retrospectives”, author Nelson provides a ‘comprehensive retrospective’ which considers a mix of process-based and outcome-based measures of project success. [1]
Project Success Criteria
I believe this model is very applicable to Information Systems and I will briefly outline each criterion according to how I would use each one to measure the success of an information system.
According to the article, the process-based criteria include:
1. Time – Whether the information system project commenced and was completed on time and whether it was carried out according to the planned schedule.
2. Cost – This involves looking at whether the information system was completed within budget. If not, did it exceed the budget by a small or large amount? A project can often be deemed successful if it exceeds budget provided it met other measurable criteria.
3. Product – This involves looking at the actual system produced and deciding if it was of acceptable quality and met expected “requirements, usability, ease of use, modifiability, and maintainability”. An information system can be completed on time and according to budget and may satisfy basic functions but if the product does not satisfy user and other requirements it may be deemed a failure.
The three outcome-related criteria include:
4. Use – Whether the information system is actually being used by its target users and is fulfilling its intended purpose.
5. Learning – This looks at whether the information system “increased stakeholder knowledge and helped prepare the organisation for future challenges”.
6. Value – This looks at whether the information system resulted directly in improved efficiency and/or effectiveness for the intended organisation. Measuring the value can include methods such as Net Present Value, Internal Rate of Return and the balanced scorecard.
Using all 6 of the above criteria can help an organisation to measure the success of an information system. However, for the criteria to be of use, stakeholder goals or expectations need to be agreed and clearly defined before the information system is developed and/or implemented. In my previous blog I talked about how different stakeholders will have different expectations regarding a project and different views on success. Nelson also notes that “success is in the eye of the beholder” and notes how “a project may be subject to review by a host of stakeholder groups, including the project sponsor, system users, project team, maintenance and support personnel, internal and external auditors, and top management.” Thus, a project may receive an entirely different opinion on success depending on who’s measuring it so it’s imperative to agree on goals at the outset.
[1] Nelson, R. (2005) Project Retrospectives: Evaluating Project Success, Failure, and Everything in Between. MIS Quarterly Executive Vol. 4 (3) 361 – 372
Leave a Reply