Impact of User Acceptance Testing (UAT) phase on Organisation Agility
Date posted: 2nd August 2023
User Acceptance Testing (UAT) is a product development practice that is very popular with traditional methods of building products. These traditional methodologies are characterised by long development phases and deployment of the product into a UAT Environment followed by an elongated period of testing in the UAT environment.
The question is whether UAT will continue to be a relevant pratice for agile teams and their stakeholder on the premise that organisations adopt agile ways of working to be able to build better products sooner. The UAT phase is a practice that is often mandated by Senior Managers within the organisation and because the UAT phase cannot fit within the Sprint (in the case of Scrum Teams), these team works around the “impediment” by tweaking the Definition of Done for their product to exclude UAT. In other words, the Scrum Team tends to amend their Sprint deliverable to a product that is development complete but not yet tested by the users and/or stakeholders.
For the scenario that is painted above, regardless of how fast the Scrum Team works to get their work items to the dev complete state, the Scrum Team effectively builds up a batch of work that isn't released for some time. The UAT phase that could last from 4 weeks to 3 months; and we believe the UAT phase is a disservice to any efforts to build up organisation agility capabilities.
Agile Frameworks such as Scrum help organisations and product teams manage complexity, and implementations of these frameworks should not introduce additional complexity. Some examples of additional complexity introduced by a separate UAT practice includes:
How do teams fix defects that are found during the UAT phase?
Would this affect the ability of the team to focus on the work on the sprint?
Are defects fixed by a subteam of developers?
How would the teams merge the increment from the UAT phase and the increment from the sprints?
At Valuehut, we support our clients to integrate all forms of testing within the Sprint (including User Acceptance Testing); the promise of Scrum is helping teams deliver usable and valuable products to their users as quickly as possible and by building capabilities within the team to get User Acceptance Testing done as part of the Sprint, the team has a greater chance to deliver a valuable and releasable product to their Users / Stakeholders within every sprint.
There are 3 patterns and associated approaches that we have followed to helping Scrum teams eliminate the UAT phase from their product delivery methodology.
Test environments that do not represent the Production Environment: Scrum teams that do not have access to a test environment that is a representation of the production environment would usually have a UAT environment, which is shared by so many teams. Non representation could refer to the size and capacity of the environment or an environment that is not set up properly with all upstream and downstream applications. To fix these issues, the team should make a continuous case for the environment that represents the production environment; make transparent to the management the cost implications which could be opportunities lost to elongated lead times.
Unknown Test Cases and User Scenarios: Oftentimes, business users that are expected to execute User Acceptance Test have created a set of User Scenarios and Test Cases that have not been shared with developers. In this case, the Scrum Team should make a case using the cost of rework and partner with business users so that these User Scenarios are shared with the Scrum Team.
Unavailability of Test Data in the Test Environment: In this situation, the business is usually not comfortable loading representative test data in the test environments due to a number of reasons, such as incapable test environment or data privacy concern. Developers should work to anonymise data and build test harnesses that consistently allow the Scrum Team access to representative test data.
It is worth noting that eliminating the UAT phase takes time and requires the Product teams to partner with the business leaders over a period of time, running little experiments per time until the UAT phase is made redundant. The only way to make the UAT phase redundant is to gather enough empirical evidence that no defects are found in the UAT phase; this would build trust with the stakeholders over a period of time, and the scrum team should provide the stakeholders with the evidence of tests run within each sprint.
Organisational Agility Capability helps organisations to be effective (building the right products e.g. product that customers love to use) and efficient (building the products right e.g. reducing time to market); as a result, organisations that want to compete in a complex world need to rethink practices such as the User Acceptance Phase that slow the overall time that it takes to get new products or features into the hands of their customers.
If you would like to speak to us about working with you to improve the lead time for your products, then contact us today.
Photo by UX Indonesia on Unsplash
