Under the decentralized governing structure, the projects use several tools and adopted several processes, methodologies and technologies inconsistently for developing software products which come under same domain. But the organizations need governance to standardize on a framework comprising process, tools, and resources that would help them save time and reduce product/process failures in order to bring in the results that may likely be in the best interests of the organization. A project is intended towards the success if it involves stakeholders in each stage and has a good planning, aware of any kind of risks, has good estimations, involves quality team, has a good communication among the stakeholders throughout the software development, and finally the project should have qualified management. Recommendations from the investigation into the failure invited us to restructure the IT services team, implement an Information Systems Strategy for any IT organisation, instigate independent monitoring procedures for IT development projects, and improve communication and build trust between the IT organisation and Customer.
Projects that fail or cancelled typically show one or more of these characteristics:
Schedule slippage
Cost overrun
Scope delivered is not the scope expected by end users
Key functionality is not delivered or does not work properly
Sometimes requirements of the project are met, but expected benefits could not be achieved
Good risk management and contingency planning not in place
A proper change management does not exist
Implementing a complex new product may be the most difficult and risky effort an organization is facing today. With a more project budget, the organisation will identify a project team headed by an experienced project manager to design and build the system and hold them responsible for delivering on their contractual commitments. But they have to deliver a solution that meets customer's objectives and for this, the project team should have experience and capability to ensure that the project will succeed. If the staff has limited experience in the required technologies, methodologies and in management, failure is not an option. Poor quality management can lead to rework, customer dissatisfaction, higher costs, and missed deadlines. There is a need to apply an effective framework that must be prudently applied in order to identify, anticipate and address these risks before they cripple the project. With all that we have on the line, there must be always an unbiased advice on the potential risks to the project and to the organization.
COALESCED QUALITY MANAGEMENT FRAMEWORK
The vast majority of projects surveyed did not utilize sufficient business analysis skill to consistently bring projects in on time and budget. While interviewing executives / project managers the one's most likely to respond are those that believe that things can be improved. It seems to us as if the call is for more accurate testing processes to insure that each phase and component of the project meets quality and performance standards. So, there is a need for risk reduction strategies to be outlined, well-considered and should be followed by organizations implementing IT projects. Coalesced Quality Management Framework (CQMF) is a recommended risk-based framework which is a collaborative environment, fundamentally in accord with best practices for avoiding failure of IT projects. The proposed framework will address the above problems with earlier detection & Prevention of errors either in the product or in process thus bringing down the total cost of quality.
The following are the goals of the proposed framework
Help the projects to complete the work within the established schedule, cost and effort.
Reduce the test time.
Increase the productivity and improve the product quality.
Increase the test productivity by reducing the rework.
Conduct thorough quality reviews and inspections thus allowing the reduction of test effort.
Prevent 'defect leakage' in all common frameworks and development lifecycles
Provide standardized way to document or compare projects.
Overcome excessive project delays due to not enough key resources.
Clear executive-level view of the project portfolio and its business impacts.
Policy Formations
Management Reviews
Metrics Collection and causal analysis on outliers
Escalations to Senior Management
Facilitating project for the implementation Quality Management System
Periodic Audits
And NCRs / Observations follow up
Periodic Reviews
Defect Analysis
Quality GroupA Theoretical Model Of CQMF
The perception of today's Project Managers with regard to Quality Assurance activity as a no value-added function and also the process models used mostly today like agile methodology stress upon lean process where inspection should be minimized or eliminated. This is justified as long as the developing process is well understood, everyone does their job the way they should, all the features work the way they should, etc. But this is rarely the situation in many companies. This is not a problem with the principles of lean. It's a problem with the implementation of it and with people who don't understand what they are doing. CQMF is a more improvement-oriented approach which expands on the focus of IV&V evaluations and establishes a model of collective intelligence and consulting environment to improve customer service and optimize software development/service processes. Large software companies today are increasingly started realizing the value of good project management for the execution of their projects in order to ensure quality project planning, execution and performance at appropriate cost. Effective oversight is critical to good project management and Independent Validation and Verification (IV&V) is an integral part of it within the project life cycle. To ensure this, the industry must adopt better practices for software development. The organizations needed better governance and needed to standardize on a framework comprising process, tools, and assets that would help them save time and reduce service-level failures.
Relevance Of Implementing IV&V Process In CQMF
In a custom development projects, the needs of the customer are relatively static whereas, in a product development situation, the needs of the "marketplace" are constantly changing. So the risk of failure due to an incorrect understanding of the project's goals is unavoidably higher for a product development project than they are for a custom development project. In many environments the SQA and SQC (Verification/Validation) form an independent group (Quality Group) separated from an application development team which reports to a project manager if the group is attached to a project or to a quality Head/Manager if it functions as separate group in an organisation. Figure 1 shows the activities performed by quality group in any organisation. Quality management system only conforms to existing quality standards by monitoring the work products for conformance to standards and processes are monitored for conformance to procedures. A quality assurance representative appointed by the quality group ensures that the project follows the defined processes and reports the metrics to the organizational repository. The QA representative conducts process compliance audit/reviews to highlight areas that are not following the defined process for the project. By adhering to comprehensive quality management system and quality assurance processes, it could be possible to leverage project experiences and learning to bring about predictability in processes and continuity & sustainability in quality if this system is controlled and run by the right people. But most of the times the SQA/SQC activities end up in corrective action by
Figure1: The activities of Quality group
analysing the origin of the defects, examining the SDLC to determine where the defect was introduced (Requirements or Design or Coding) and reviewing these with the project managers and other associates for possible improvements. This
type of approach will not work for large projects or critical projects. CMMI defines what processes and activities need to be done and not how these processes and activities are done. The function of SQA is to assure that the standards, processes, and procedures are appropriate for the project and are correctly implemented at the project level to produce required internal and external deliverables. In CMMI SQA and SQC play a major role but not an exclusive role as both do not get involved in building any of the products but are only involved in Verification and Validation activities or collecting metrics for measuring the effectiveness of the processes in order to implement continuous process improvement. There are many reasons for cost overruns. The less visible one is the underwater part representing 'Poor quality resulting in significant rework, 'cost overrun due to high rework and idle times, and customer dissatisfaction.', These are the ones that cause the sinking of 'Titanic' rollouts unless there is a system of studying below the surface and safeguarding against what is beneath the surface [1].
So, IV&V is conducted parallel to, but separate from, the system software development activities. IV&V apart from doing assessment activities like reviewing, analyzing, testing, and monitoring should act as technical advisors to help a project manager oversee a project by handling the project's schedule, budget, deliveries, etc. IV&V group is composed of experienced domain experts, in-depth technical expertise with strong communication, management and planning skills for assessment, analysis, evaluation, review, inspection, and testing of software products and processes. The CQMF framework shown in figure 2 ensures that the end product meets the cutomers' business objectives and also significant improvements in cost, schedule and quality for these organizations. In this framework, various stakeholders are involved in the development of a system where business analysts and domain experts collaborate in requirements gathering/designing a system which caters to a set of specifications. These specifications are verified by various stakeholders (Domain Experts, IVV team, users etc). Traditionally the system requirements document has been a static document capturing the requirements of a client The process of defining requirements and its verification operates in a feedback loop till a defect less final design document is arrived at. Process repository has detailed checklists, standards, templates and guidelines exist within the processes to bring in rigor and predictability into every aspect of project planning and execution. These processes and templates exist across various types of projects (reengineering, maintenance, development, package implementation, etc) represents every aspect of the project (requirements analysis, change/ configuration management, tailoring, defect or schedule estimation, etc). Projects use project metrics, process metrics and a process-capability baseline from process repository to estimate effort, schedule tasks and predict defect levels during the project-planning phase. The process database is based on data from past projects and ensures that project plans are realistic. These metrics increases the effectiveness of the implementation of the Project Control Decision Support System and also project monitoring by Quality group and project managers based on these metrics increases its effectiveness. Further, project monitoring based on these metrics increases its effectiveness. The CQMF serves as organisation's "eyes and ears" into the inner workings of the project and independent of the project team and the customer, it provide insight into all aspects of the project: requirements management, adherence to the schedule and budget, project governance, technical architectures and change management with the involvement of IV&V pool, subject matter expects, groups like portfolio management office, quality group, and process/product/tool repository. More and more organizations are realizing that they must be able to wire solutions together more quickly to get to market rapidly and be competitive. SQA which maintains and enhances the repository, based on the experiences gained from the project implementation and bench marking against international practices.
Senior Management / Project Stakeholders
Projects
Project Management Office
Custom/Product development projects at onsite/offshore
Portfolio Management Office
Project Control Decision Support System
Subject Matter Experts Pool
Mathematicians, Microbiologists, Chemists, security professionals, etc
Project Knowledge Base
1. Domain Checklists
2. Domain models
(Reusable)
3. Best practices
4. Generic Architectures
5. Project Portfolios
6. Domain specific training
materials
Quality Group (SQA, SQC, & SEPG)
IV&V Pool
Dedicated experts - Business Analysts, Technical Architects, Domain Experts, Testing Analysts, and Tool specialists
Process Repository
1. Process Templates & Checklists
2. Standards & Procedures
3. Project metrics
4. Process metrics
5. process-capability baseline
Tool Repository
Tools - Testing tools, Audit tool, estimation Tools, Product/Process/Portfolio Decision Support tool, etc
Reusable Products
Figure 2. Coalesced Quality Management Framework at organisation level with centralized IV & V pool , Project Control Decision support system, Process/Product/Tool repository, Subject Mater Experts Pool, Quality Group and Portfolio Management Office
Involvement Of Expert Pool In IV&V
According to new research, success in 68 percent of technology projects is "improbable". Poor requirements analysis causes many of these failures, meaning projects are doomed right from the start [2]. In many organizations, requirements are not detailed enough to enable project to make needed changes and get to the end goal reliably. Key to the success of the design and development of a software system like bio-informatics software or an e-security solution mostly depends on the obtaining requirements from different people with expertise in different competencies. E.g., developing bio-informatics software requires the services of a Biology and/or bio-chemistry researcher to provide domain related requirements and test the software, a mathematician to provide guidance towards optimal algorithms, few computer scientists to develop the code etc.
This case study is taken from e-Security area. The requirement for application security in today's industry is so pressing that any company that does not immediately align, would be left far behind in the game and will lose its competitive edge. Involving the domain experts like Mathematicians, computer scientists and legal professionals. The involvement of the legal experts is an interesting thing here. As the developed security product should be in compliance many legal policies and government regulations, there was a need for the project team to consider the inputs from legal experts while deploying the product and issuing the digital signatures certificates which are used for authenticating digitally signed electronic records. Many organizations and institutions have been changing their functions from simple service or product offerings to Innovation oriented activities. Innovation always needs the involvement of experts in various domains. This has been explained above through an example. Also the knowledge delivered by the domain experts during the execution of innovative projects should be captured in a procedural way and a Knowledge Database (KDB) has to be maintained and updated for future purposes.
Domain experts review the existing business case for applicability and ideas and develop a list of functional descriptions covering not only the specific business need but other user needs and future possibilities that could impact or might be impacted by the project. Domain experts, with possible assistance by functional analysts and the development team draft the key Use Cases, capturing all of the necessary user requirements. The IV&V team will ensure that all stakeholders (Domain experts, Functional Analysts and users) assess possible misunderstandings, overlaps, holes, and other inconsistencies and organize these requirements or Use Cases from which they develop a comprehensive and detailed user requirements document which specifies the desired functionality of the system. Technical architects develop the Technical Specifications, which combine the user requirements from the domain experts, the strategic standards and technologies from the information specialists, and the tactical approach to system development recommended by the CQMS framework. A reuse program is an organizational structure and collection of support tools aimed at fostering, managing, and maintaining the practice of reusing software in an organization. Current methodÂologies and procedures do not inÂclude reuse as part of their process. The IV&V provider will verify that all stakeholders have bought-in to all changes which impact project objectives, cost, or schedule.
FOCUSING BEYOND PROJECT MANAGEMENT
During the development of software products, a project manager must make decisions based on uncertain predictions and many interacting factors influence the development including allocation of resources, skills and technologies required, methodology to be followed (e.g Waterfall, JAD, Spiral, Agile etc) and many other depending factors. A good decision-making by the project manager requires a mixture of skills like identification of options, clarity of judgment, firmness of decision, and effective implementation for managing large projects. One of the ultimate goals of any businesses is to deliver what customer expects and even exceed the customer satisfaction. For this it takes a lot of planning, analysis and consultation. The problem with most of the project managers is that they focus on their individual projects, not on the previous successes of other projects. Normally, the decision making is much more localized and it is of more responsive to prevailing circumstances. Project Managers usually start with a new way of doing things based on intuition and insight or they may be the early adopters of leading-edge technology and using the new concepts to build systems. If unexpected events occur, though the Project Managers have the autonomy to try alternatives, may result in costly later rework and rediscovery. It is always better to analyse and follow the advocated concepts, and knowledge of the experts' practices that entered the mainstream of software engineering. Above all, the level of confidence placed in the hypothesized conclusions is dependent on the availability and reliable knowledge and the experience of the project manager.
Promoting Leverage Of Reusable Project Information
Project portfolio is a knowledge-based data useful to be able to monitor and control the mix of various types of projects that enables managers to become aware of all of the individual projects in the portfolio, and provides a deeper understanding of the collection as a whole. The portfolio concept may be recognized in many organisations, but portfolio data are not centrally managed and/or not regularly refreshed. Organisations obtain only about 60% of the value that could be derived from their businesses and the remaining 40% of available value is lost, reportedly, due to errors in decision-making and weaknesses in business systems [4]. The Project Management tools being used may have good data display and management capabilities, but as the schedule and cost overruns are still common, and the risks of project failure remain large, we can understand that the results from the tool potentially mislead the decision makers. Effective PPM (Project Portfolio Management) enables us to know what benefits to expect from a project and to track the realization of those benefits as the project progresses [5]. Project classification scheme has to be established and are classified in many different ways which include size, type (e.g development, maintenance, conversion, testing, innovation), geographic location, skills or technologies required, client or market served (domain based), and methodology (e.g Waterfall, JAD, Spiral, Agile etc). No one approach is best for every organization. This classification will help to understand knowing the various categories to which a project belongs and yield information most useful for decision making.
Promoting Accountability For Project Investments
The above classification also helps to standardize project information template that are useful to write and evaluate proposals in a consistent manner. By focusing on specification, estimation, management, and control, software development teams can deliver help organizations and projects within a particular domain to build applications with flexible architectures, enabling project managers to predict more accurately the effect that the project will have on the business so that helps to respond to market changes and other demands very quickly. The systems must also help organizations analyze the impact from a financial and service level, and be able to respond. Today, the businesses know what they want to do, and the technology is there to help them. But to move forward they need governance, a disciplined approach, and the ability to reuse their assets and maximize their ROI.
Need For A Decision Support System In Projects
There is a perception that it is not possible to achieve all aspects of quality because of interrelationships. Software Quality Assurance engineers and Project Managers must determine, based on experience, what trade-offs are to be made within the specific project since each project has different objectives. To make the best trade-offs, all relationships must be understood and use experience to interpret the impact. For this an organization has to come out with a model that acts as a decision support system that collect, record, process, store, communicate, retrieve, display, and disseminate information to the projects so that resulting approach provides for delivery of a timely and quality system within costs. Sometimes there is need for the projects to take several trade-off decisions such as the decision to use COTS software products as opposed to developing custom software or reusing software components, or the decision to use an incremental delivery versus a complete, one-time deployment etc.
A key focus of a project decision support system tool is to help projects in decision making with the following objectives:
Overall collection and managing of organization's project portfolio
Identify project risks early and manage them before they become problems.
Deliver quality systems which meet or exceed customer expectations when promised and within cost estimates.
Deliver systems that work effectively and efficiently within the current and planned information technology infrastructure.
Deliver systems that are cost-effective to enhance and maintain.
Provide project management with appropriate information and ensure that the project is effectively managed throughout its life cycle.
Ensure that the system requirements are well defined and subsequently satisfied.
Continue to identify and mitigate risks, coordinating with the business area to ensure that new technology can be phased in.
The Problem With Rule Based Expert Systems
Large projects are difficult to manage and some people take them to success. Everyone has vast experience in facing the problems brought up in project life cycle. When people solve problems, they frequently are reminded of previous problems they have faced. Though the project portfolio manager (rules-based system ) establishes the rules and procedures for making portfolio decisions, where a set of rules are repeatedly applied to a collection of facts [3], mostly we use rules for the most common situations we come across and also encounter many situations which violate or are outside the bounds of the generalizations we make. Much work in artificial intelligence, for example, is done in "decision support systems." These systems are based on the notion that expert knowledge consists of a collection of rules.
Let us consider a rule based decision support system using rules arriving at a decision as to which model to use based on the characteristics of requirements as one of the criteria.
IF: (AND (requirements easily defined and well known)
(requirements be defined early)
(requirements change frequently))
THEN: (conclude Waterfall)
The other criteria being the Characteristics of the Project Team, Characteristics of the Users, and Characteristics of the Project Type and its Risks. By formulating the rules with the help of an expert in a particular domain, we may simulate expert behavior in that domain. Not surprisingly, decision support systems have run into a significant problem when there is a "missing knowledge" and are brittle. When faced with a problem, they fail because they are not grounded in cases. They are unable to fall back on the details of their experience, find a similar case, and apply it. Likewise, they are unable to use similarities between tough problems and previous experiences to update their rules. In a practical scenario, sometimes end-users often present project team with a sketchy set of initial requirements and the designers build a prototype with the client, test it, refine it and test it again until the client is satisfied. Sometimes the project team lack even the most rudimentary set of user requirements at a beginning of the project and, thus, rely upon an on-the-fly, code-and-fix exploratory process. When time and funding constraints weigh heavily COTS is used. The most harmful misconception of the people to make system intelligent is that from knowing a lot of rules.
So, it is also important to understand the complexity of the problem to be addressed and how many variables must be dealt with. If the problem is fairly complex with a number of measurable response variables and the rule based system fail to behave properly due to the missing knowledge and the complexity involved in re-examining the entire system of rules. Our research has shown that over-analyzing a situation is not as effective as sensing, analyzing, and responding to change in a timely way and thus in an decision support system, if a rule is wrong, it is worthless.
3.3.2 Case-Based Reasoning (CBR) Technology
Software process reuse involves different aspects of the knowledge obtained from generic process models and previous successful projects. The benefit of reuse is reached by the definition of an effective and systematic process to specify, produce, classify, retrieve and adapt software artifacts for utilization in another context. The Case-Based Reasoning (CBR) technology is used to manage the collective knowledge of all projects within an organization by aggregating project data into a central database, something never before captured in quite this way, and use it to suggest better representations. The focus is formalizing the framework for evaluating and prioritizing projects and on implementing tools and processes for supporting project budgeting, risk and issues tracking, requirements tracking, and resource management. Cases contain and relate individual bits of knowledge about instances of things you have experienced [6]. There is no doubt that intelligence comes from knowledge, but most of that knowledge looks quite a bit different than what we find in a textbook. It is his practical experience that leads one to become an expert. Project Managers constantly experience such reminding, comparing one experience to another so as to learn from both. So, much of human reasoning is case-based rather than rule-based. Problem solving by the project managers requires organizing the cases of their experience (generalizations) and representing the knowledge in a useful form, so they can be recalled when relevant..
CONCLUSION
The ultimate goal of CQMS framework is to create a process-oriented mind-set within the organisation and always make the project stakeholders stick to its commitment and consistently deliver quality software with the involvement of IV&V. Since software development process is evolutionary and is connected with risk analysis at each stage of the project, the project stakeholders have to learn how to respond and manage risks. Maintaining a balanced portfolio is essential in managing the risk and resource allocation. Also there is a need for AI based systems (DSS) As today's software systems are significantly large, complex and critical, that only through the establishment of a world-class collaboration framework and automated approaches the projects can eliminate risk assessment phase saving our time in software development. Though major IT companies are working rigorously to automate project portfolio and software engineering activities, but lot of research has to happen to automated software engineering techniques by combining both rule-based and case-based reasoning techniques and augmented with causal or mathematical models, statistical studies, or world knowledge that justifies the knowledge base during reasoning.
