Archive | IS Success RSS feed for this section

Framework for Evaluating IS Success

22 Feb

Group 2 Members:

Christine Coughlan, Clifton Moore, Dermot Lucid, Niamh O’ Farrell &
Ronan Murphy

Introduction

We have created a framework which allows an organisation to evaluate the success of an information system unit. In order to develop our framework, we researched a number of IS success models developed by key authors in the IS field, for example, DeLone & McLean, Sedera, Gable, Seddon and Nelson. In researching these models we have identified both value and flaws within these models and have developed our own framework based on what we think are the most important IS dimensions to evaluate when measuring the success of an IS unit for any organisation today.

Our framework identifies key dimensions which must be measured to evaluate the overall success if an IS unit. Any firm, large or small, can use our success framework to measure the success of their IS unit by choosing suitable metrics to measure each dimension contained in the framework. We have created a framework that we believe to be both flexible and customisable. In our framework we have identified possible metrics used to measure each dimension but it is up to an organisation to decide on the most appropriate metrics to suit their organisational context and IS strategy. It is imperative to choose appropriate and agreed measures or this framework will fail to deliver its potential value.

* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *

Framework for Evaluating the Success of an IS Unit

IS success framework

Click on the image for a better view

* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *

Explanation of Dimensions

[1] Context

Seddon et al (1999) presented the Cameron and Whetten 1983 (Fig 1, Below) framework for contextualising and evaluating Organisational Performance, adapted in Seddon et al (1999) to IT Effectiveness. [1] We have adopted the same seven-point framework to contextualise IS Success, as endorsed by Petter, DeLone, and McLean (2008) and outlined in the earlier post ‘Relatively Successful IS’.

Though all seven points are important, we suggest, in line with Seddon et al, that
1. Stakeholder Perspective? 2. Type of Information System, and 7. Against which referent is Success to be judged? are central to contextualising Success. The Context dimension of our Success framework is designed to establish, and justify, what is to be deemed ‘Successful’ from the standpoint of the stakeholder concerned, regarding the relevant IS system, and in the particular situation/organisation. To this end the Context aspect should be regarded as a Canvas to identify and outline the perspective or varying perspectives from which the analysis is based.

Stakeholder

“A stakeholder is a person or group in whose interest the evaluation of IS success is being performed” (Seddon et al, 1999). Seddon believes that due to a range of different individuals within an organisation they are going to evaluate IT success in different ways and perspectives. The inclusion of the “stakeholder” section in the framework is aimed to provide an organisation with the tool to adapt and understand the view of a projects success from all stakeholders’ views. Each stakeholder may use their own dedicated canvas, or if deemed useful, multiple stakeholders may outline their perspectives/concerns on a shared canvas. As is the case with Osterwalder’s Business Model Canvas, participants might post their views into the various categories using ‘stickies’, perhaps colour-coded to their individual stakeholder perspectives etc, to build up a visual representation of where their various priorities lie. In either case, this approach allows for comparison of stakeholders various perspectives, priorities and concerns, and will lead various parties to a more complete understanding of the success/weaknesses of the system in question. For example, a user might acknowledge a manager’s concerns for cost and the IT department’s concerns over reliability, versus their own concerns regarding usability, and vice-versa.

The elements ‘Timeframe’, ‘Type of Data’, and ‘Purpose of Evaluation’, are important for clarity, while acknowledging whether the system is for ‘Voluntary or Mandatory Use’ is a key factor to keep in mind within the backdrop to the evaluation. In the operational canvas these four elements (shaded) might be replaced by more relevant concerns, and so, should be regarded as suggestions. Once the vision of success is established stakeholders can turn their attention to the Quality & Impact sections and prioritise and even assign weighting to the various underlying dimensions. The relevant, prioritised/weighted dimensions can be measured using a Likert Scale against Sedera et al’s 27 corresponding measures Figure 3 (Below) as mentioned in the earlier post ‘ IS Success Canvas’.

The table borrowed from Seddon et al (1999), Fig 2 (below) contains examples of various stakeholders and information systems, and this table can be employed to inform the context dimension of the framework. The table column and row headings are useful as prompts but are not exhaustive of potential perspectives. However, the strength of the canvas approach is that it is blank and can therefore accommodate all stakeholder/perspectives and various information systems. Also, though informative as regards Stakeholder and IS type, we favour Sedera’s refined four-dimensional model and its tested measures (Fig 3, below) over the measures contained in Seddon’s table (Fig 2, below)
In a nutshell, the left side of the framework is a canvas to establish and outline what is to be deemed IS success. The right side of our framework is concerned with evaluating the IS against this established vision of success.

[2] Quality and Impact

The DeLone and McLean original IS success model classified measures of success into six constructs; System-Quality, Information-Quality, Organisational-Impact, Individual-Impact, Satisfaction, and Use.
Gable et al (2008) later proposed that information quality and system quality as identified by McLean and DeLone should be elements of a greater construct – IS Quality, while individual and organisational impact should be sub elements of an IS Impact construct.
Furthermore, Gable et al proposed that the ‘Satisfaction’ and ‘Use’ concepts as identified by DeLone and McLean should only be used as a metric to measure IS Impact and IS Quality and should not be treated as dependent constructs.
Thus, in our framework we considered both models and have confined the 6 constructs identified by DeLone and McLean into 2 key constructs as put forth by Gable et al; IS Impact and IS Quality. These constructs can be seen on the top level of the diagram.

The Impact construct is concerned with the eventual outputs delivered by an IS. The reason organisations invest heavily in information systems is because they expect the IS to have positive impacts on individual users and the organisation as a whole. Individual-Impact looks at how the IS has influenced the productivity and capabilities of individual users. Possible measures which can be used include individual productivity, learning and decision effectiveness.
Organisational-Impact is concerned with how the IS contributes to overall organisational results and capabilities. Business process change, cost reductions and overall productivity can be used to measure organisational impact.

The Quality construct is used to measure the IT-Artefact or technology element of IS.
Information-Quality is concerned with the quality of the information produced by the system, for example in reports and on-screen. Some measures which have been developed and successfully measured according to gable et al (2008) include importance, relevance and accuracy.
System-Quality measures the success of IS from a technical and design perspective. Tried and tested measures of system quality include reliability, flexibility, and potential for customisation.

Underneath Quality and Impact in the diagram we have the structure of the IS unit and Net Benefits.

[3] Structure of IS Unit

The structure or make-up of an IS unit can greatly impact its success, for example the level of commitment and support from top management, the quality of communication, culture and the skills of the employees. We will explain each of these to give a greater view of how the structure of an IS unit can influence IS success.

Top Management Support

It is extremely important that top management do not forget about a project after the planning stage but instead are commitment at the time of system implementation. By being directly involved in a project, top management guides the implementation team, allocating resources for projects, and stepping in to solve critical issues likely to affect implementation.

Communication

Management of an IS unit also affects communication within an organization and ultimately the productivity of users. Communication in an enterprise is vital in managing a company more efficiently, keep close monitor on strategies, strong relationships with employees and to have strong relations with partners/clients.

Culture

Culture within an organisation is also critical in determining success as it can impact how innovation affects IT practices and overall performance. Culture can impact organisations in the following three ways
1) Culture within an organisation can provide unwritten guidelines for employees in how to create a good workplace and strengthen relationships in order to improve the social system in the organisation.
2) Culture in an organisation can also affect the ability to deal successfully with issues from both internal and external integration.
3) It can also determine the differentiating between in-group and out-group people.

Employee Skills and Training

Employee skills being one of the most important factors within an organisation are critical in achieving success. If the employee does not meet the requirements/skills needed to carry out the required tasks, it can affect productivity and efficiency. It is also important that a business has a well-established training program for new employees in order to gain the appropriate skills that may be required specific to the company.

[4] Net Benefits

As a group we felt that Net Benefits is needed within an IS framework to support management teams in determining the success of their IS unit. The Net benefit dimension was also used in the DeLone and McLean model (2003) for organising IS success measurements. [ ] Net benefits are the extent to which IS are adding to the success of individuals, organisations and groups. The support management team needs to identify what their net benefits are. Examples of organisational net benefits may include: improved decision making, productivity, increased sales, reductions in cost, profits, economic development and creation of jobs, (DeLone & McLean, 2008).

Conclusion

Our framework is a synthesis of the key dimensions evident in the IS Success model research, and we feel that the framework is applicable or adaptable to all IS evaluations. The framework is intentionally open in nature with regard to its dimensions and measures making it ideal for quickly establishing and explaining across various stakeholders the success or less successful aspects of a system, while if necessary, thorough quantitative methods may be applied to the various dimensions, depending on the nature of the evaluation.

References:

• Seddon, P. B., Staples, S., Patnayakuni, R., & Bowtell, M. (1999). Dimensions of information systems success. Communications of the AIS, 2(3es)
• Petter,S., DeLone,W. & McLean,E. (2008). Measuring information systems success: models dimensions, measures, and interrelationships.
• Gable, Guy G. and Sedera, Darshana and Chan, Taizan (2008) Re-conceptualizing information system success: the IS-Impact Measurement Model. Journal of the Association for Information Systems.
• DeLone, W. and McLean, E. (2003). The DeLone and McLean Model of Information Systems Success: A Ten-Year Update.

Figure 1: Seven Questions to Answer when Measuring Organisational Performance – Cameron and Whetten (1983)

Figure 1: Seven Questions to Answer when Measuring Organisational Performance – Cameron and Whetten (1983)

Figure 2: IS Effectiveness Measures used for different combinations of Stakeholder and System – Seddon et al, (1999)

Figure 2: IS Effectiveness Measures used for different combinations of Stakeholder and System - Seddon et al, (1999)

Figure 3: Gable et al (2008) Impact Measures

Figure 3: Gable et al (2008) Impact Measures

Context Adaptation of D&M IS Success Model (2003)

21 Feb

Introduction
In researching IS success models for our individual blogs, it was clear to see to all that the framework or model put forward by DeLone and McLean in 1992 and revised in 2003 was the basis for the majority of other academics when they were creating their own IS success framework. With that in mind and since the majority of our blog posts were either about the DeLone and McLean model or related to it, we decided to use that as our foundation.

Context Adaptation of D&M IS Success Model (2003)
Capture

Culture
Information systems success within an organisation can depend on the culture within the organisation and that’s why culture is the first dimension on our framework as mentioned in AndrewFitz22’s blog “How culture can impact on information systems success
According to Ismall [1] in regards to information systems, culture is one of the determinants in its success. Also culture can impact on how innovation within the organisation can improve information technology practices and overall organisation performance.
Culture impacts on an organisation in the following three ways
1. Provides unspoken guidelines to employees on how to get along with each other in the organisation and enhance the stability of the social system in the organisation.
2. Enabling the ability to deal successfully with problems from internal integration and external adaption.
3. Culture also enables differentiating between in-group and out-group people.

Organisational Structure
Decision Making Structure: There can be two sides associated with implementing a proper decision making structure. If a company decides to choose a decentralisation process studies have shown that it is “one of the strongest facilitators for the adoption of customer-based inter-organisational system and IT system in large and complex organisations”. Studies have also shown that there are benefits of a centralised organisation. Results have highlighted that a “better management effectiveness for end user computing” exists and also that there is a greater chance of successful “strategic information systems applications” (Hussein et al, 2007). These both in turn would have an impact on IS Success.

Goal Alignment: This factor outlines the effect of linking up both Business and IT goals. If both these goals are aligned and have a clear direction in which they both are headed it can help in achieving IS Success.

Resources Allocation: This factor categorises resources into money, people and time. These three should then be managed properly by ensuring that sufficient money is available, the right people are in place and there is enough time to complete a IS project thus in time helping to lead to IS Success.

Top Management Support : “Top management support is defined as: devoting time to the [IS] program in proportion to its cost and potential, reviewing plans, following up on results and facilitating the management problems involved with integrating ICT with the management process of the business. “It’s important to consider the role which top management support plays in Information System success. Despite its relevance, “Practitioners and Researchers alike have focused their attention o factors they can more directly control and appear to only pay lip-service to TMS.” Young and Jordan, (2008), examined IS projects as due to their high costs it is critical for organizations to achieve success. “An issue demanding board level attention because of the high levels of investment and the strategic consequences of failure.” The work of Young and Jordan, (2008) highlight the importance of TMS, and declare it the most important critical success factor. While Young and Jordan present a concise argument in favour of TMS as the ultimate CSF, there are additional organizational factors that play a role in determining IS success.

Management Style: “Management style deals with the way in which management tends to influence, co-ordinate and direct people’s activities towards a groups objectives.” Hussein et al, (2007). A review of the literature would suggest that management styles impact on communication within the organization, and ultimately on the productivity of the users. “Lu & Wang’s study (1997) seems to support the major finding in Igbaria et al.’s study(1990) on the relationship between leadership style and user satisfaction. Both studies found that leadership style and system success are correlated significantly and positively.

Managerial IT Knowledge: Another organizational factor which is important to consider is Managerial IT knowledge. “Managerial IT knowledge refers to senior management experience and knowledge concerning information technology.” Logically it makes sense for senior management to understand the technology behind IT projects, in order to realize the benefits an organization can accrue through successful implementation of IS. Research has revealed that the IT knowledge of senior managerial staff directly and positively impacts on IT utilization, Boyton et al (1994). Therefore it is in the interest of organizations to employ those with a background in the related field. A manager who is aware of the technology, value and risks may then manage the project accordingly.

Relationships
Our model has removed the direct relationships, drawn as arrows in the D&M model, between the six dimensions of IS success. Instead, lines connect five dimensions; service, information, and system quality, use, and user satisfaction. This is intended to demonstrate that these five dimensions interact and ultimately impact upon the net benefits of the information system, shown by a double arrow. The arrow is double sided to indicate that these benefits feed back into use and user satisfaction, like in the D&M model (2003). The arrows have been removed given the lack to support to date for the direct relationships between the dimensions at an organisational level. Neither is the model intended to be a process or causal model explaining the determinants of success, but instead is aimed to communicate that each of these dimensions impacts upon the success of an information system. If a system is not deemed a success, through an IS balanced scorecard for example, then each dimension can be examined to identify in what area the system is weak. Although this may satisfy practitioners, academics will still question the relationships between the dimensions of IS success.

Dimensions of IS success
When examining the success of a system, it appears to be necessary to look at more than just the benefits that the systems can offer. Considering the six dimensions of IS success present in the D&M model (2003) appears to provide a more complex evaluation of IS success and identification of elements of the system that are weak. As advised by Petter et al. (2008) success measurement programs can be attached to each dimension of the system together with weights depending on the relative importance of each.

There are six dimensions of IS success: system quality, information quality, service quality, system use, user satisfaction and net benefits. System quality is the ideal characteristics of an IS. Information quality is the ideal characteristics of an IS outputs. Service quality is the quality of the support received from IS. The three qualities would influence the using of the IS and affect user satisfaction. By using IS companies can achieve net benefits such as improving decision-making, increasing sales and so on. On the other hand, companies decide IT investments according the measurement of net benefits, which would affect the other dimensions in turn.

And Finally …
We found that the six dimensions set out by Delone and McLean in their 2003 model to be the most complete approach to measure the complex nature of IS success. We felt, however, that some elements were missing, namely organisational and cultural factors which all impact on the success of an organisation’s IS system.

IS Success Canvas?

10 Feb

Success is a lousy teacher. It seduces smart people into thinking they can’t lose.  –  Bill Gates

IS Success Canvas?

The DeLone & McLean IS success model was instrumental in identifying the key dimensions of IS success, and remains robust in most regards, as outlined in Petter et al (2008). However, Gable et al (2008) have excluded the constructs “Use” a consequence of IS Impact and not a dimension, and “User Satisfaction” which is measured indirectly under the other variables.

Fortunately, for any practitioner seeking to evaluate IS success within an organisation, it is not necessary to invent the wheel. Many valid measures have been identified tested and refined. Sedera et al’s Multi-dimensional IS Success Instrument for example, takes into account many of the concerns outlined in Seddon (1999), and has collected and tested success measures, combining them into a comprehensive 27 measure IS Success Instrument from an initial 37 items in total.  –  (See Gable et al (2008) for list of measures.)

In my previous post (Relatively Successful IS) I presented some of the concerns raised with regard to type of IS, stakeholder perspective and organisational context etc. in determining IS success. In particular, the context matrix presented in Seddon et al (1999) stands out as identifying the challenges and providing direction in tailoring an evaluation to a particular perspective of success. However, it remains doubtful that all of these contingencies will ever be entirely controlled for.

The challenges of relating an (ideal) IS success model concept to a practical context are reminiscent of Osterwalder, A., Pigneur, Y., & Tucci 2005, and so many others’ quest to establish the dimensions of the “Business Model”. While in academia the research continues, in practice, Osterwalder’s “Canvas” provides a framework, informed by the research, into which various users/stakeholders from various contexts etc., can input their own specific content. I would suggest designing a framework much like the Business Model Canvas, whereby all perspectives etc, can be accommodated. The IS Success Canvas framework would be informed by the knowledge gained through DeLone & McLean, Seddon et al, and Sedera’s modeling and research.

Success consists of going from failure to failure without loss of enthusiasm.’ –  Winston Churchill

References

Gable, Guy G. and Sedera, Darshana and Chan, Taizan (2008) Re-conceptualizing information system success : the IS-Impact Measurement Model. Journal of the Association for Information Systems.

Petter, S., DeLone, W., & McLean, E. (2008). Measuring information systems success: models, dimensions, measures, and interrelationships. European Journal of Information Systems, 17(3), 236-263

Seddon, P. B., Staples, S., Patnayakuni, R., & Bowtell, M. (1999). Dimensions of information systems success. Communications of the AIS, 2(3es).

Osterwalder, A., Pigneur, Y., & Tucci, C. L. 2005. Clarifying business models: Origins, present and future of the concept. Communications of the Association for Information Science (CAIS)

The Problems of IS

10 Feb

Communication problems between users and designers may be the major factor which leads to the failure of IS. The users and IS specialists have different backgrounds, priorities and goals. The specialists often have a highly technical or orientation to problem solving. They look for elegant and sophisticated technical solutions in which hardware and software efficiency is optimized at the expense of easy of use or organizational effectiveness. Users, on the other hand, prefer systems that are oriented to solving business problems or facilitating organizational tasks. Often the orientations of booth groups are so at odds that they appear to speak in different tongues.

In general, IS problems can be classified into: design, data, cost, operations.
DESIGN
– The system fails to capture essential business requirements
– Information may not be provided by system quickly enough or in a usable format
– Some systems go unused because they were designed with a poor user interface
DATA
– The data in the system have a high level of inaccuracy, or incompleteness
– The data may not be broken out properly for business purposes
COST
– Some systems cost so much to implement or operate that they cannot be justified by the demonstrated business value of the information they provide
OPERATIONS
– Information is not provided in a timely and efficient manner because the computer operations that handle information processing may be break down
– These problems can be caused by technical, managerial, and organizational factors

Source from:

http://www.ann.ugal.ro/eco/Doc2010_2/Rosca_Banica_Sirbu.pdf

Case example 2 of applying the D&M model

10 Feb

The second case study is of a large high profile business called ME electronics that maintained their success by using the Delone and Mclean model. ME electronics is an electronics retail chain who’s main target customers are wealthy consumers by selling expensive branded goods. ME electronics have used a website for the last 4 years which supports a broad communication strategy and serves as a one-way communication channel to the customer used for broadcasting company information. However at ME electronics customers are unable to place orders through their website. They face large competition that has integrated this service into their company already. Me electronics new strategy is based on electronic customer relationship management (ECRM). The goal is to integrate an interactive website that can track all customer interactions. [1]

Capture c2

Customer satisfaction is one of the main objectives in increasing sales per customer. The aim is for customers to complete e-mail based customer satisfaction surveys that will measure system quality, information accessibility of the customers and ME’s responsiveness to customer e-mail inquiries and overall satisfaction with the on-line experience. It is also necessary that ME collect data on the highest products that are interacted with by customers and correlate interactions with sales activity. [1]

We have seen from the above examples the flexibility and relevance of the updated Delone and Mclean model and the theoretical foundation it provides for a business. It also provides a guide for empirical success however it is important to note that it is not a fix solution that any company can use and expect success. The D&M model is quite vague and it does lack strategy but it’s an affective guide and a matter of which business can utilise the framework most affectively is what will determine success as seen in the examples.

 

 

[1] DeLone, W.H., and McLean, E.R. Measuring e-commerce success: Applying the DeLone

& McLean information systems success model. International Journal of Electronic Commerce,

9, 1 (2004), 31–47.

Case example 1 of applying the D&M model

10 Feb

From the many blogs concerning the Delone and Mclean model including zonic89 and mcoconnell [1],[2], we have now gained a strong theoretical understanding of the framework. However, what I have personally noted about the D&M model itself and its varied and extended models is that they are quiet vague. One must ask can this framework be applied for any type of organisation. In this blog I will be talking about two case examples of both a large high profile business and the other a small retail store.

Barnes and Noble is an example of a low profile business that succeeded by using the Delone and Mclean model. Barnes and Noble main competitor is Amazon who’s growth in sales and market value highlights the value of the online model. Customers could order from their own homes with a larger selection than any other large bookstore with lower prices. Barnes and Noble decided to compete against Amazon by using the updated Delone and Mclean framework. [3]

Capture case1

The main measures of success for Barnes and Nobles online book sales business are sales revenue and investors reactions to the company’s “success” as shown in the market valuation. To take into account all net benefits it is necessary to measure the quality of the users experience and the consumer’s satisfaction of usage with the system. It is vital that the Barnes and Noble website must be easy to use and available whenever the customer wants to access it. It is necessary that the information displayed must be relevant to the customer’s interests. If e-mail response to purchase transactions is not sufficient then a call centre should be available. The frequency number of site visits and their satisfaction can be recorded through repeat purchases. [3]

As we can see from the Barnes and Noble example, the D&M model is an effective method of improving a business through the six dimensions. It is a useful method/guide for a business that needs improvement by improving the sub factors for each dimension as seen in the diagram.  However the model isn’t a solution but more of a guide as it lack strategic direction. It is how a company interprets the model best and its dimensions is what will distinguish it from other firms in order to achieve IS success. [3]

 

[1] https://sopinion8ed.wordpress.com/2013/01/11/is-success/

 

[2] https://sopinion8ed.wordpress.com/2013/02/08/relationships-in-the-dm-model/

 

[3] DeLone, W.H., and McLean, E.R. Measuring e-commerce success: Applying the DeLone

& McLean information systems success model. International Journal of Electronic Commerce,

9, 1 (2004), 31–47.

 

Building Successful Information Systems

10 Feb

It is difficult to define a successful information system, but in generally, a successful IS should be accurate, reliable, used widely and work as intended. The whole quality of the organization should be promoted with a good IS. A successful IS will do the following:

  •         Realize the business goals of users;
  •          Create the corresponding value for the company with proper costs;
  •         Meet the defined standards;
  •          Have accurate and reliable output;
  •          Be easy for users to learn and use;
  •          Be flexible.

Building IS

The reasons to build an IS now are not just for saving money, reducing work force or being efficient but supporting decision making, meeting the high expectation of customers and coordinating groups in an organization. The following is the process of developing an IS:

X[{F9ZCBE}SM{62RC`ECA{O

Figure 1. The IS development process

The factors influencing the IS development can be classified into external environmental factors and internal institutional factors. The external factors include constraints and environment opportunities. The costs of resources, competitions with others and the changes of government regulations can be regarded as constraints. The opportunities include: new technology, new resources, new government programs and so on. The institutional factors will influence the adoption and design of IS through the values, norms and interests which the organization will choose.

In detail, these factors should be included:

  •          Support of top-level managers;
  •          Involvement of all the users;
  •          Use of a proven systems development methodology;
  •          Clear goals and objectives;
  •          Focus on the most important problems and opportunities;
  •          Good training programs;
  •          Well-organized maintenance programs.

Source from:

http://www.ann.ugal.ro/eco/Doc2010_2/Rosca_Banica_Sirbu.pdf

Relatively Successful IS

10 Feb

“If A equals success, then the formula is A = X + Y + Z.  X is work. Y is play. Z is keep your mouth shut.”

–          Albert Einstein

Reification of success.

Academic researchers are predominantly concerned with the general, generalisability, objectivity and universality of models etc. Whereas, practitioners are dealing with the specific, where there exists the entire range of different systems, in different organisations, of different types, with different issues and problems/contexts, all from the perspectives (subjective) of the various stakeholders’ various expectations of success and failure.

Gable et al (2008) observed that  ‘….60% of studies employing Delone and McLean constructs use a single construct, with over 90% using 3 or less. This is not to suggest that any specific study employing less than the full set is flawed – one would have to consider the specific intent of each study (in fact we do not advocate the full set).’

‘According to a study by HEO & Han (2003) the constructs of the IS model have different degrees of importance depending on the firm’s characteristics…’ Peter et al (2008)

‘The importance of analyzing IS success at multiple levels within organizations has been discussed among academics for over a decade [Cameron and Whetten, 1983, Leidner and Elam, 1994, Quinn and Rohrbaugh, 1983, Sedera et al., 2006, Tallon et al., 2000, Thong and Yap, 1996, Yoon, 1995]. The concern is that different employment cohorts have differing experience of the system. Yet, IS studies have, in the main, attempted to quantify the impacts of IS by analyzing data collected from only a single employment cohort.’ Gable et al (2008)

Seddon et al (1999) highlight these and many other issues regarding the application of success models to both research and practice. Here they attempt to categorise the applicability of the various success measures according to the type of system, and the stakeholder concerned. ‘Seddon’s [1997] re-specification of DeLone and McLean’s model posits that different individuals are likely to evaluate the consequences of IT use in different ways: “IS Success is thus conceptualized as a value judgement made by an individual, from the point of some stakeholder”’ Seddon et al (1999)

Seven Questions to Answer when Measuring Organizational Performance [Cameron and Whetten, 1983]

1. From whose perspective is effectiveness being judged?

2. What is the domain of activity? (depends on tasks emphasized in the organization, competencies of the organization, and demands from external forces)

3. What is the level of analysis? (individual, subunit, organization, population, societal)

4. What is the purpose of evaluation?

5. What is time frame is employed? (short, long)

6. What types of data are to be used? (objective or perceptual)

7. Against which referent is effectiveness to be judged? (effectiveness of this organization compared to: some other organization; some ideal level of performance; stated goals of the organization; past performance of the organization; or certain desirable characteristics)

Seddon et all (1999) based their research framework around Cameron & Whitten’s seven questions (above) for measuring organisational effectiveness. Clearly, these concerns are as relevant to IS success as they are to organisational effectiveness, and as outlined, go some way toward revealing the relative nature of IS success, and in particular, question seven – substitute Information System for Organisation and we get:

7. Against which referent is Success to be judged? (Success of this Information System compared to: some other Information System; some ideal level of Success; stated goals of the Information System; past performance of the Information System; or certain desirable characteristics)

It follows that any framwork for evaluating organisation-specific IS success must first take these concerns into account. In other words, any evaluation of the success of an organisation’s IS  unit must first establish the relative understanding of success.

“I don’t know the key to success, but the key to failure is to try to please everyone.” –  Bill Cosby

References

Cameron, K.S. and D.A. Whetten, (1983) “Some conclusions about organizational effectiveness”, in K.S. Cameron, and D.A Whetten, (eds). Organizational Effectiveness: A Comparison of Multiple Models, New York: Academic Press, pp. 261-277.

Gable, Guy G. and Sedera, Darshana and Chan, Taizan (2008) Re-conceptualizing information system success : the IS-Impact Measurement Model. Journal of the Association for Information Systems.

Petter, S., DeLone, W., & McLean, E. (2008). Measuring information systems success: models, dimensions, measures, and interrelationships. European Journal of Information Systems, 17(3), 236-263

Seddon, P. B., Staples, S., Patnayakuni, R., & Bowtell, M. (1999). Dimensions of information systems success. Communications of the AIS, 2(3es).

IS and Success

8 Feb

“I don’t know the key to success, but the key to failure is to try to please everyone.”  – – Bill Cosby

 

IS projects have a notoriously high failure rate, with many abandoned before completion. When an IS unit gets to a stage whereby it can be observed and measured – this in itself is a considerable success. However, the majority of projects that do reach implementation are described as challenged, – overrunning their original estimates, failing to meet objectives, or failing in terms of user acceptance. In fact, over the past decade, only 30-35% of IT projects were deemed truly successful.

Image

Yearly IT project success/failure rates – Standish Group

The table (above) and the figures in the previous paragraph are sourced from Standish Group’s “Chaos Report”. The Standish Group reviews thousands of projects and produces a very popular though controversial (It is questionable how representative their sample is) yearly report outlining IT project success and failure rates.

So, how does Standish determine success?

Success = “The project is completed, on-time and on budget, with all features and functions as initially specified

At first glance, this criteria for IS success, though project-orientated, seems plausible. However, a system may conceivably be completed, on time and on budget, and as initially specified, and end up hidden under the stairs because nobody really wants to learn yet another new system after all. (The criteria fails to cover all dimensions of IS success.) Conversely, various commentators argue that many projects fail to meet some or any of these requirements but would still be considered successful. Indeed, there is a latent expectation that complex projects will over-run time and budget and may not deliver all that is hoped, or deliver differently than expected.

The example outlined (above) is IT project-orientated. Nevertheless, it serves to highlight the flaw in the IS success debate. The difficulty with IS & Success is that success is Relative.

“If A equals success, then the formula is A = X + Y + Z.  X is work. Y is play. Z is keep your mouth shut.”

– Albert Einstein

References:

http://blog.standishgroup.com/

Relationships in the D&M model

8 Feb

If the D&M model (2003) is used as the basis for our model of IS success, the features within it needs to be addressed. The main components are the dimensions and the relationships between them. It seems that all of the dimensions in the updated model should be included though the context within which these dimensions operate should be considered; IT leadership for example [1]. External factors also need to be addressed [2]. The relationships that exist between dimensions have been addressed by Sabherwal et al. (2006) and Petter et al. (2008) and this appears to be the most complex feature of the model. Furthermore, the strength of each relationship in the model is not fully understood.

From my interpretation of the model, the relationships between the dimensions of the model provide us with an understanding of IS success determinants, either as a process or causal model. What it does not show is if success has been achieved, which ultimately a practitioner will be interested in. At an organisational level insufficient data exists for supporting the relationship between net benefits and other dimensions with the exception of system quality (see diagram) [3].

Petter et al. (2008)

Petter et al. (2008)

So, maybe the relationships could be removed from the model and then success in each dimension could be focused on rather than the determinants of success. A model which identifies in which dimensions an IS system is successful or not might be useful to practitioners. Petter et al. (2008) have already advised practitioners ‘to deploy success measurement programs that incorporate all six dimensions of IS success’, attach relative weights to each dimension, and also consider an IS balanced scorecard to measure net benefits [4].

References
1. Sabherwal, R.; Jeyaraj, A; and Chowa, C (2006), Information System Success: Individual and Organizational Determinants, Management Science, 52 (12), 1849-1864; ronnoc90, https://sopinion8ed.wordpress.com/2013/02/05/top-management-support-a-key-factor-in-is-project-success/; cmcoughlan, https://sopinion8ed.wordpress.com/2013/02/05/is-strategy-leading-is-unit-to-success/
2. mcoconnell, https://sopinion8ed.wordpress.com/2013/01/30/a-3-d-model-of-is-success/
3. Petter, S.; DeLone, W.; and McLean, E. (2008). Measuring information systems success: models, dimensions, measures, and interrelationships. European Journal of Information Systems, 17, 236-263; Sabherwal et al. (2006).
4. Petter et al. (2008).

%d bloggers like this: