Building the business case for application packaging and testing automation

Enterprise IT organizations are under enormous pressure to do more with less. Mounting transformation project workloads brought on by organizations moving to business-as-usual IT (e.g., Windows OS Servicing, hardware refresh cycles, moving to virtualization, maintaining existing applications, and rolling out new applications) often do not come with an increased budget or more resources. 

From an application management point of view, constant updates are leading to packaging teams becoming bottlenecks and application owners not testing their applications properly (or even at all!) to rush things along. Consequently, there is an increased risk of security vulnerabilities, business disruption due to applications not being deployed correctly, stalled projects, increased spending to prevent projects from getting stuck, or increased headcount to help clear bottlenecks.

If this sounds like you, you are not alone. In fact, recent Juriba surveys showed that

• 92% of enterprises and other large organizations have an End User Computing (EUC) strategy to move their applications into the cloud (native, Intune, or Azure Virtual Desktop (AVD)),
• 89% of organizations identify application compatibility issues as their top IT change bottlenecks, and
• 86% of organizations are demanding more automated application management and testing.

We all instinctively know that application packaging and testing are tedious, labor-intensive processes and that automation is the answer, but many don't know how exactly to justify it. Today, I want to walk you through 7 easy steps that will allow you to create a bullet-proof business case—making it easier to sell it to management. Because, let's face it, automated packaging and testing simply make sense. 

[Click here to watch our on-demand webinar about building your business case for automated App Testing and Packaging.]

1. Estimating your preliminary budget

Probably one of the most important aspects of any business plan is a sound financial case. It is important to compare the real packaging and testing costs you encounter today by managing the process manually, compared to what it would cost you to automate the process.

If you do not know your application packaging and testing costs per application per annum, you can take an industry average. We have found for enterprises with a large portfolio consisting of 1500-3000 low-, medium-, and high-complexity applications, it will cost an average of $680 per application per year for packaging and smoke testing when done manually.

You need to adjust the cost if you:

• Run multiple projects. The cost above is referring to packaging and testing an average application once a year for a Windows OS Servicing update. If you are running additional projects (e.g. moving to AVD with MSIX), you will need to multiply that cost.
• Outsource. If you have outsourced your application packaging and testing, the cost will be defined in a contract.
• Have complex applications. Since your annual application management cost largely depends on the volume and complexity of your applications, we also recommend taking an inventory of your applications and adjusting the cost depending on whether or not you have an overly complex application portfolio (e.g., you have a lot of custom or internally build applications). 

We usually recommend that our customers determine the number of low-, medium-, and high-complexity applications and multiply these with the estimated/actual cost to calculate the app management budget. 

2. Understanding your business case parameters to get a fully loaded cost

Now that we have established our preliminary budget estimation, we need to define our cost parameters, i.e., how many and which types of resources do we need to manage as part of this process. This will give us baseline numbers to estimate what the fully-loaded costs of your application management function would be and allow us to compare the manual versus automated costs. Here at Juriba, we typically split the resourcing analysis between your internal IT application packaging and testing team and those external resources that are also critical to the process:

Internal application management function would include:

• Application packagers*,
• Application testers*,
• Scheduling resources to coordinate the creation of the virtual machines or delivery of the images, and 
• Application managers.

Additionally, we have external resources to consider:

• External application owners (business unit) and 
• Application coordinators (business units, coordinate pilots with, etc.).

*Day rates obviously vary greatly from region to region and often depend on the size of the contract, but we have found that you have to calculate, on average, $350-$550 per day per packager/tester. Oftentimes, application packagers and testers are outsourced, which means you will need to look at your contract with the service integrator. Within an outsource contract, you also may have a retainer that includes a certain number of applications that are included per year or per month (e.g., 50) and any application on top will have a per-app price.

3. Define your team

Next, we need to understand the roles and responsibilities. Essentially, we need to know which and how many resources will be involved in what way.

• What's going to be suitable for automation and what isn't? With automation, and depending on your application complexity, about 70-80% of the applications right out of the box will be suitable for automation and can be self-serviced by the application owner.
• What is the new volume for the packaging & testing team? Automation will reduce the number of apps the internal team will have to deal with to about 20%. This, of course, means that complex applications remain for manual work, and these will need to be serviced by experienced packagers and testers.
• Can you re-utilize the headcount for higher-value activities? By freeing up resources, you can now re-utilize that headcount for more value-added activities, like coordinating and capturing the application tests script captures with the business owners and the application owners.

4. Define your application logistics

Next, we need to define the scope of the project. The word "project" is used here loosely and refers to anything from your app management tasks for a specific IT Transformation to defining the scope of your annual app management as part of your BAU/Evergreen IT. In order to understand the flow of the process, we need to understand:

• What are my application logistics resources and budget?
• What is the criticality of my applications?
• What type of testing do the apps need to go through? Is the smoke test enough? Does it need an automated functional test?
• Do certain applications always require a manual sign-off?
• Is it always going to require a test in the field against existing machines before we can move forward?
• What platforms are we moving to? You will need to assess the suitability of the apps for those platforms and therefore the users and devices to move to them.

The answers to these questions will determine how much time is needed for each app. We also need to understand how the process is going to be managed from a logistical point of view. Who's going to manage it? What input do we need from the business to support it? Ultimately, we need to think about what each rollout will look like, how we can make it repeatable, and how we can make it most effective (e.g., use ring planning concepts). 

5. Choose the right automation solution

There are many application packaging and testing tools out there, with different tools specializing in specific aspects of the process, or having limited application coverage. To get the biggest return-on-investment and to future proof your investment, we highly recommend choosing an automation solution that:

• Is fully integrated into your Workplace Automation platform (like Juriba DPC and Application Readiness suite) to enable you to automate the entire process from end-to-end and do velocity planning based on application readiness,
• Integrates with your existing request processes (e.g., tools like ServiceNow, as well as existing application/desktop management tools, such as Intune and Microsoft Endpoint Manager),
• Automates the entire process end-to-end from the first application request to application retirement, and
• Takes care of the heavy lifting. If used with Juriba, you can take advantage of our self-service capabilities as well as automated email communication which eliminates the heavy lifting in your project.

6. Calculate time & resource savings with automation

Now that you have established what your current application packaging and testing expenditure is, you will need to contrast it with the time, resource, and therefore cost savings you can achieve through automation! We have found that, on average, our customers see the following savings:

Time and resources needed to test 500 applications:

• Manually: 750 hours or 42 days (at 90 minutes/app) with 2 people 
• Automated: 50 hours over the weekend (7-14 minutes/app) with no human interaction required (93.33% faster!)

Time and resources needed to package & test 4 simple, medium, & complex applications (12 Total) to corporate standards:

• Manually: 91 hours 
• Automated: 53 Minutes (96.8% faster!)

7. Consider other (soft) benefits

While the significant cost/time savings as well as headcount reduction/resource reallocation are huge, it would be remiss of me not to mention the huge advantages that come with moving to automation. They might not necessarily be qualifiable in the traditional sense, but they often add bigger value in the long-term and should be included within your business case presentation:

• Being proactive rather than reactive. Being able to be proactive about managing your application portfolio allows you to not only save on licensing and maintenance costs (as you are managing your estate much tighter), but also to drive digital transformation, determine suitability to move to the cloud or virtualization, and much more.
• Modernizing your application portfolio. There is a huge interest in moving to Azure Virtual Desktop (formerly Windows Virtual Desktop) but you have to convert your application portfolio to MSIX and sideload them with app attach *.VHD files. By automating your application packaging and testing process, those aren't a problem anymore.
• Minimize risks. Application packaging and testing is a very manual-labor-heavy process and therefore prone to human error. Repeatable, standardized, and automated test scripts and packaging procedures eliminate those errors and therefore drastically reduce associated risks. The outcomes are robust and trustworthy. 
• Ability to do velocity planning.  By having a centrally managed inventory, you can test the applications first that would have the biggest impact on your project timeline rather than randomly working through a list. This significantly accelerates your transformation timeline.
• Performance testing before and after.  With Juriba's sophisticated automated testing, you can not only find out if an application works or doesn't, but you can also understand how it is performing and compare the performance tests to any snapshots before and after.
• Future-proof your investment. Your investment in application packaging and testing automation will not only enable you to streamline your application management for this upcoming project but for your Business-as-Usual (BAU) functions in the future. 
• Buy-in and support from the business units. Because automating makes the entire process more repeatable and therefore more predictable, business units are often much more supportive as they know what to expect. While this might mean some upfront time with the IT team, for example for test capturing, it is much easier on them in the long run.
• Less testing. By alternating the smoke test and the functional test, the app owners do not need to do anything other than just review the results and video of the test running through.
• Productivity gains. When done manually, it takes a long time to request an app, run a discovery process including source media collection, write out the install instructions, and package the app. By being able to self-service the process, app owners can massively improve their productivity because you get your application out in the environment within hours rather than weeks and months.
• Reduction in Shadow IT. Your automated application packaging and testing are going to be so efficient and fast, that you will see a reduction in business units or groups going rogue because you are able to get the applications rolled out, packaged, and tested much faster than they can manually.

Final thoughts & recommendations

To sum up, make sure that the numbers stack up and get as many of your own "real" numbers as possible. Always ensure that the business fully understands and buys into automating the process. To do this, it helps to make the process for everyone involved as slick as possible. Be sure to measure exactly which savings and productivity improvements you are driving. It is always a great idea to add to your business case a section on how you are going to measure this improvement going forward.

If you found this helpful, why not register to watch the full webinar on-demand where we will go through each of these areas in more detail.