User Acceptance Testing (UAT)

User Acceptance Testing (UAT) is usually known as the end-user software testing performed before introducing a new system to an organization. The main objective of user acceptance testing (UAT) is to ensure that the new system performs the tasks expected to perform and meets the business requirements. During user acceptance testing (UAT), the completed system is tested against the user needs for the first time.

In the case of User acceptance testing (UAT), formal testing is required concerning ‘user needs, requirements, and business processes.’ There is no specific document for this; however, it focuses on user needs and goes beyond testing software to include business processes. One more aspect is acceptance criteria that have to be satisfied, and it defines what is acceptable to the users.

Why Testing is required 

There are multiple reasons of requirement non existence and imperfection which include:

• Sometimes for the end users, it becomes very difficult to think or imagine what the requirements are going to be and what the system should evolve over time
• Most of the time, end users or sponsors cannot communicate the requirements even when they understand them clearly. Consequently, the intended requirements are not reflected in the requirement specification. 
• Even if the end users or sponsors’ intentions are reflected clearly by the requirements in the requirement specification, requirements may be interpreted in a wrong way by the developers. 
• However, the requirements are thought, written, communicated and interpreted properly, still the developers commit mistakes while writing the program code.

Process of User Acceptance Testing (UAT)

The user acceptance testing (UAT) process consists of various stages that include:

• Recruitment of/provide training to UAT team
• Set up and plan for UAT
• Design tests
• Implement tests
• Evaluation
• Decision making
• Support
• End-user training and user guides

Stakeholders in User Acceptance Testing (UAT)

There are multiple stakeholders during an implementation. The stakeholders are identified and intimated at every stage of the project progress. For user acceptance testing (UAT), the intended needs of some of the important stakeholder groups should be considered which include:

• Sponsor

The sponsor, a group of people that have paid for the system. The sponsor basically pays for the new system to be implemented. He can be the owner of a startup or an executive-level personnel in a large company who possesses the finance for the new system to be implemented. The sponsor always focuses on the usability and value for money of the newly implemented system that the people will use and he makes sure that the users will get the maximum benefits from the system.

• Business Managers

Business manager(s), a person or people who are accountable to deliver the business interests that are expected out of the system implementation. They need to be assured that the newly implemented system is available for use and efficient enough to deliver the stated benefits.

• Quality Manager

One of the most important management roles involved in the User acceptance testing (UAT) exercise is Quality Manager. The quality manager is primarily focused on the quality of the delivered system. And a test manager is interested in the overall quality and efficacy of testing, and responsible for advising and guiding the user acceptance testing (UAT) team to get the best user acceptance testing (UAT).

• End-users

 The end users who use and operate the system implemented and its benefits.

• Development and Quality Assurance Teams

The development and quality assurance/testing teams responsible for the actual implementation will cooperate during the user acceptance testing (UAT).

In user acceptance testing (UAT), all stakeholders are responsible to deliver specific information relevant to each group.

User Acceptance Testing (UAT) Team

Based on the project size and the organization, the user acceptance testing (UAT) team can consist of the following members:

• Programme Manager

A programme manager is accountable for delivering multiple projects.

• Project Manager

A project manager is accountable for delivering a project.

• Project Management Officer (PMO) or Administrative Staff

A project Management officer is accountable for managing the user acceptance testing (UAT).

• Test/Quality Manager

A test/quality manager is accountable for managing all testing including UAT.

• User Acceptance Testing (UAT) Team Manager

A user acceptance testing (UAT) manager is accountable for delivering user acceptance testing (UAT).

• User Acceptance Testing (UAT) Trainer

A user acceptance testing (UAT) trainer is accountable for providing user acceptance testing (UAT) training.

• Business Analyst

A Business Analyst is accountable for documenting business/project requirements. The user acceptance testing (UAT) team is primarily responsible for providing adequate information to the major stakeholders to make a reasonable decision on whether the system is acceptable or not. The user acceptance testing (UAT) team has the key responsibility to execute testing and provide reports on testing to the stakeholders.

The UAT teams have varied sizes and a user acceptance testing (UAT) team can consist of any number of individuals starting from a single person to many. In user acceptance testing, analysis and testing will be important and have specific roles during UAT support and make it more useful and effective.

Key Roles in a User Acceptance Testing (UAT) Team

Business analysts (in-house or outsourced)

During a project’s implementation it is likely to have miscommunication between the business and the technology due to varied responsibilities and objectives. This creates a gap between the two and Business analysts bridge this gap by translating their languages’ requirements.

Business analysts work as the representative of the ideas of each group and make sure that the newly developed system fulfils the user expectations. The business analyst defines the business requirements and translates them into technical Specification. Business analysts also coordinate with the testing team and provide their assistance in writing the test cases and test scripts. 

Based on business analysts’ availability, they generally take part in test execution and reporting. They also help evaluate the severity of incidents to fix issues found during testing.

User Acceptance Testing (UAT) Manager or Coordinator (in-house or outsourced)

The user acceptance testing (UAT) manager creates a plan for UAT with the help of test/project manager. In his UAT plan, he writes the testing goals for UAT, the testing activities, and the resources for testing needed to execute and deliver user acceptance testing (UAT) goals. During the UAT plan preparation, the UAT manager makes sure that the required data and information, testing environments and user accounts are ready for UAT and he tracks the incidents throughout the user acceptance testing (UAT).

User Acceptance Testing (UAT) Testers (in-house users)

UAT testers or the users have to test the newly implemented system as they are the ones who establish a specific connection with the system and are going to use the system in the present and future. The in-house testers or the testing team include the end users and subject matter experts (SMEs) who can deep dive into the system and help the UAT as they have the system’s expertise. The in-house testers are also supposed to take part at the time of business requirements definition and evaluation in the beginning as they are going to be the end users.

UAT testers have the right to present their ideas of process improvement and process updates whenever required along with the test scripts execution and incident logging, and finally give their feedback about the user experience. 

UAT Team Skills, Knowledge and Experience

The user acceptance testing (UAT) team is required to perform the UAT process and its operations independently. The UAT team must be able to plan and manage their test schedule and set up testing standards to ensure that the UAT is not going to be executed as per any specific interests and schedule of a stakeholder. The UAT team is considered to possess some particular skills, knowledge and experience that include:

• Business requirements thorough knowledge and analysis
• Knowledge of system architecture and recent fixes
• In-depth knowledge of user acceptance testing (UAT)
• Understanding and creating user acceptance testing (UAT) plan
• Assigning relevant resources and taking the right decision on issues escalation
• Knowledge and understanding of formal testing methodology
• Ability to represent the user groups
• Proactive in gathering the right and relevant documentation to match with the outcomes
• Detail oriented and thoughtful to be able to think about alternatives to processes that need alteration
• Positive way of providing feedback to subordinates
• Knowledge and understanding of test automation tools
• Familiarity with general software and agile working environment

Each team member has his own importance in the process and contributes to the user acceptance testing (UAT) with the help of other team members to get the best results. Working in a team instead of working individually makes it easier for all the team members to get faster results and achieve the stated goal. A good team is required to follow six actions to ensure the maximum chances of getting success which include:

• Purpose

The purpose should be stated distinctly and all team members must understand the importance of a team, its existence and essential goals. They need to make sure that the goals are in line with the business and project to get the maximum benefits for the business.

• Participants

The participants should be willing to participate and contribute their valuable time and efforts to accomplish the objectives. They should take the responsibility of their work to show commitment towards the achievement of objectives.

• Communication

Communication among the team members should be clear and open. Team members should be encouraged to discuss the issues openly in a friendly environment and provide their honest feedback to other team members irrespective of the nature of feedback. Team members should also be open to accept their feedback whether it is positive or negative.

• Knowledge and Skills

The team members should have the right skills and knowledge to make the right decisions that would be helpful in achieving the team’s goal. An ideal team with the right people and relevant knowledge and skills has the potential to accomplish their objectives in the best effective manner. And, the team members are able to put up their opinions to be heard and resolve any conflicts in an efficient way.

• Creativity

The team members should welcome and encourage creativity. Team members should be able to make good use of their creative thinking that is important to expedite change.

• Decision and Agreement

In a team, decisions are made and those decisions should be in agreement of all. The user acceptance testing (UAT) manager or coordinator is considered to intervene when the teams involved in decision making and the disagreed do not give their opinion on that. It helps in understanding the objections from the teams, to consider them and make right decisions based on all opinions.

User Acceptance Testing (UAT) Preparation

To plan user acceptance testing (UAT), the following steps need to be followed:

• Decide what to achieve

Prior to writing a full proof testing plan and committing any resources for testing, we need to ensure that what needs to be tested and if it is the right thing we are testing. Before the start of testing, we have the deliverable from the development team and need to make sure few things to start with which include:

• An Updated Business Requirements Specification
• Testing performed by testers and developers
• A system that is ready for User Acceptance Testing (UAT)
• Trained User Acceptance Testing (UAT) team
• Testing environments and accounts set up

• Acceptance Criteria

Acceptance criteria are the standards that are set by the sponsors and users that can be considered as a target to be acceptable at the conclusion of UAT. The realistic acceptance criteria could be:

• The newly implemented system does not have any defects
• The newly implemented system functions correctly
• The newly implemented system is available to launch on the release date

• User Acceptance Testing (UAT) Objectives

User acceptance testing (UAT) is performed for the following reasons:

• To reduce the risks
• To boost the confidence
• To evaluate the availability of the system to be live
• To ease the transition to live phase

These are the primary objectives of user acceptance testing (UAT) to be included in the UAT strategy and tracked towards the acceptance criteria. 

• Entry Criteria

The system code keeps changing and thus it invalidates the tests due to change and would have to be repeated. For this reason, the entry criteria is used for UAT to give it a fresh start. The realistic entry criteria could be:

• Overall testing up to system testing is done without any pending incidents
• Critical defects are zero
• Serious defects are not more than 5
• Routine defects are not more than 10
• All issues that affect the testing are resolved

• Testing Definition

A user acceptance testing (UAT) strategy needs to be defined to understand what to test and a plan needs to be created to execute. Now the testing skills are used to decide on how can the objectives be achieved in an efficient way. If a project is really simple and has a short time frame, strategy and plan can be ignored and acceptance criteria can directly be used to define tests.

Note: UAT strategy and planning will be discussed in detail in the next article.