Most of our customers are currently in the process of deciding 1) which Windows 10 Upgrade Management strategy to pursue (6, 12, 18 or 24 months), and whatever they are choosing, 2) how to achieve the velocity they need to get everything ready and tested to roll everything out in the defined period.
To manage your Windows 10 upgrades efficiently and with maximum velocity, there are a lot of upfront activities that have to be done first, e.g., getting a new build image ready, testing your hardware, planning your deployment rings, and more.
"Friendly Testers" Approach Does Not Lead To Optimal Results
As you might already know, Microsoft is currently suggesting to have four deployment rings: the preview ring, targeted ring, broad ring, and critical ring. In order to define those rings, most organizations used to initially look at the devices and find friendly business testers that we could bring into these rings.
This allows us to gather feedback from testers that do not mind some associated inconvenience, as to whether or not the applications work. Unfortunately, people are starting to realize that this process doesn't always work out as planned because there are no guarantees that the testers even did anything with the applications on their new release, and there is no way of actually measuring the feedback on those applications.
Often testers might sign off with a simple launch of an application for example. There was also no impact analysis allowing us to determine that, if we successfully test these specific applications, we enable X devices to be ready for migration, so the friendly testing approach didn't quite achieve the coverage required to create a proper deployment plan.
Application-Based Approach For Maximum Velocity
Recently, we tackled this problem by taking an application-based approach to defining deployment rings with the goal of maximizing the velocity of the testing methodology.
The preview ring is still defined as very similar to the friendly tester approach: but now you select certain devices mathematically within each business unit and in IT, give them an in-place upgrade, and have them continue using their machine as is. Any issues will be reported back to the project team, ideally through automated application health reporting such as is available in Microsoft's Desktop Analytics platform.
What happens after the preview ring has now become a lot more formalized. Within the targeted ring, we are taking a hybrid approach: testing a huge number of applications live by picking machines that give us the most application coverage, while also testing a specific subset of applications more thoroughly by asking our official business unit testers to test some specific apps with approval and sign off.
This gives us a larger Targeted Ring, but as a result, we can plan our subsequent broad rings much more effectively. For example, if we can put these 30 devices in the targeted ring, we will have 100% of critical application coverage that opens up xx number of potential upgrade ready machines..
But in order to do that, we have to develop a holistic application testing strategy.
Four Traditional Approaches To Application Testing
In the past, we have found that there are five different types of app testing that our customers view as possible strategies for managing their app testing for their Windows 10 upgrades:
- Dedicated application testing (new machine). In this formal test, your business unit tester receives a brand new virtual machine with a new image in order to test their existing application suite or a specific application on that image. This method is more traditional and requires evidence-based documentation to ensure the test was performed accurately. It only tests the application on a vanilla build.
- Dedicated application testing (existing machine). In this formal test, your business unit tester receives an upgrade to their existing machine in order to test their existing application suite or a specific application on that new version of Windows 10. This method is also traditional and requires evidence-based documentation to ensure the test was performed accurately. The test here is against all of the applications that reside on the user's existing machine. Of course the risk here is that there is something that impacts the testing that is not common in the environment, so it is less controlled.
- Smoke test your apps in the real world. Microsoft advocate this approach and have been pushing it for a while now. Effectively a suck it and see. Roll out the upgrade to a larger, potentially undefined group of business users in your production environment. Essentially, you will monitor and look out for any issues. This testing approach can work for some well-established applications, but should only be used when you are relatively certain that the risks are minimal.
- Test your apps with an automated smoke test and functional test. This is the generally the most efficient route. Juriba partners with specific third party tooling that automatically installs and launches an application on a virtual machine, records the steps and uninstalls it, For a more sophisticated 'user' test, a functional test script can be injected after application launch that can be run against that application to confirm that the application is working according to the functional test script.
- Test your apps with an automated static analysis test. As with option 4, this is a very efficient route to understanding your application compatibility with a specific Windows 10 version. Juriba partners with third party tooling that automatically reviews the content of an application down to DLL level and ascertains based on Microsoft rules and customer experience whether the application will experience issues on the new platform.
The problem is that most enterprises cannot afford to do it all (too lengthy, costly, and labor-intensive), but they also cannot afford to do nothing at all (too risky). In order to achieve enough velocity to get through yearly upgrade cycles, they need to understand the state of their application estate and to take a pragmatic approach on the application testing cycle.
How To Determine Your Application Estate Status
While none of our enterprise customers would risk barging ahead and test none of their apps, we are finding that, more and more, large organizations do not necessarily want to fully test all of their applications. However, they do want to get a feel for which of these applications might cause problems and which ones are going to work fine.
This can be accomplished though:
- Desktop Analytics which uses Microsoft's telemetry data to flag anything that potentially is not compatible (vendor apps only),
- Static Application Analysis Partners - which allow us to analyze and assess an application package to determine whether it should work on the latest release, and
- Automated Testing Partners - which effectively smoke test applications in a vanilla environment, reporting any failures automatically.
How Dashworks Can Help
You cannot achieve deployment velocity without appropriate planning. The important thing is to categorize your applications once as either critical, important, core (part of the image, e.g., MS Office), or not important before trying to figure out which apps effectively become part of each ring. Then you can determine how, within that application estate, you will plan to test each of those apps.
No matter which methodology or combination of such (a hybrid approach is industry best practice) you choose, you still need a central repository and command and control center to manage it all.
Within Juriba Dashworks, you can create a WaaS Application Ring Coverage Report that allows you to get a quick overview of how many of your applications are critical, important, and not important, and gives you the ability to filter by project, ring name, and criticality.
.png)
By using this information, you can set up automated project readiness, scheduling and migration workflows. For example, if this app is a critical app, it needs to go through a proper UAT phase, it needs to get a sign off, it needs to be distributed slowly to people, and much more. On the other hand, if you have a low-impact application, you could create a workflow within Dashworks that will roll that app out, monitor it in the live environment, and turn it red if issues are reported.
Dashworks gives you a highly-effective and efficient hybrid approach that allows you to manage your entire application portfolio, define your deployment rings around the applications to get the most coverage, and therefore get through your upgrades with minimal hassle, resources, and time.
