As a professional that is proud of my work, I ensure that each of my installs is thoroughly “Unit Tested” by me before they go to QA – after all, the install is not ready until it can do the basics:
- Install cleanly on each approved Operating System – using Dialogs and running unattended/silently!
- Uninstall cleanly on each approved Operating System
- All registry entries properly configured based on dialog selection and input values
- All files are installed – in the correct directories, based on Feature selection
- All configurable files (XML, INI) are properly configured based on dialog selection and input values
- All shortcuts are created and functionally work
- All NT Services are created and started correctly.
- Any Web Services are created and configured correctly
Plus, realistically if it doesn’t perform the basics – what does the quality of my install package relay to the development team that is facing pressure to get their application in front of the testers – they don’t need doubts as to whether iteration #1 will be hung up with an installation that simply doesn’t install!
But what about situations where the new install will be “upgrading” previous releases that are out in the field? You have introduced a different set of testing criteria, such as:
- All ‘approved’ previously released application versions are recognized – some versions may not be approved for upgrade, so they have to be recognized and dealt with appropriately
- All ‘approved’ previously released application versions are upgraded to the new release. This includes removing any trace of previous versions, removing obsolete files, preserving user configured files.
When I perform Unit Testing on my installation package, I seem to have a handle on the iterations that must be accomplished (test with this dialog selection on, test with it off), but I also found myself needing to be able to accomplish a test and be able to compare the results with a previous results set.
What I needed was a consistent method to preserve the installed components for in-depth analysis. I started out by having a check-list, such as:
· Verify all registry entries – HKLM created
· Verify all registry entries – HKCU created
· Verify the AppPath entries are correctly created and configured
· Verify the registry entries are correctly created and configured
· Verify the services are created, started and configured as Automatic or Manual as required
· Verify all of the files are installed based on the dialog choices/Feature selection – then copy the installed file to network for analysis – be sure to preserve the Time/Date stamp!
With this checklist, I could perform multiple installs on my list of Test cases, but whenever I missed gathering something, I had to go back and recreate the test scenario just to capture the data.
Plus, I was finding that some of the test Iterations could be given to QA and reviewed for their smoke testing – because I was not gathering the data for their analysis.
Batch Archive Method
I found that I needed a method to easily gather the installed components for preservation. I ended up creating a simple BATCH Archive job that would handle the data gathering consistently and would ensure that I did not miss anything!
Typically my testing is performed under VMware’s Workstation Server and I have an open connection to the domain network. I created a BATCH job that would collect the data and copy it to a network location. If you use ROBOCOPY (available as part of the Windows Resource Kit, and introduced as a standard feature of Windows Vista and Windows Server 2008. ) you can easily copy the collected files and critically those installed directories – preserve the file attributes (Time/Date stamp).
Here is a sample batch files I created, “Forensics_Archive.BAT” – you needed to pass it one parameter, the directory subset on the network shared that I created to hold the test results. For example on \\DevShare\DevTesting\Charles\R7_Results I would create various sub-directories (NewInstall; Upgrade; etc) using the Batch Archive. For example, I could run the following:
Here is an extract of the batch file “Forensics_Archive.BAT”
IF [%1]==[/?] GOTO Syntax
IF [%1]== GOTO Syntax
rmdir /S /Q %ArchiveDestPath%\
ECHO Archiving SOFTWARE Registry Entries from installation
REGEDIT /E “%ArchiveDestPath%\ Forensics.REG” “HKEY_LOCAL_MACHINE\SOFTWARE\Forensics”
REGEDIT /E “%ArchiveDestPath%\ISI Systems.REG” “HKEY_LOCAL_MACHINE\SOFTWARE\ISI Systems”
REGEDIT /E “%ArchiveDestPath%\Uninstall Entries.REG” “HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall”
REGEDIT /E “%ArchiveDestPath%\HKCurrentUser_Forensics.REG” “HKEY_CURRENT_USER\SOFTWARE\Forensics”
ECHO Archiving SERVICES Registry Entries from installation
REGEDIT /E “%ArchiveDestPath%\DirectoryService.REG”
REGEDIT /E “%ArchiveDestPath%\Forensics Content Server.REG” “HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Recorder Content Server”
REGEDIT /E “%ArchiveDestPath%\RecorderTomcat.REG” “HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\RecorderTomcat”
ECHO Archiving PROGRAMFILES Files from installation
XCOPY “c:\Program Files\ComPlus Applications” “%ArchiveDestPath%\Program
Files\ComPlus Applications” /C /E /I /R /Y
XCOPY “c:\Program Files\ISI Systems” “%ArchiveDestPath%\Program Files\ISI Systems” /C /E /I /R /Y
ECHO Archiving Files archived during Upgrade
XCOPY “C:\Documents and Settings\All Users\Application Data\ISI Systems\Archive” “%ArchiveDestPath%\Archived Files” /C /E /I /R /Y
ECHO Archiving Information on IIS Virtual Directory from installation
cscript.exe adsutil.vbs ENUM w3svc/1/root/Forensics > IISVertDir_Forensics.txt
XCOPY “c:\inetpub\AdminScripts\IISVertDir*.txt” “%ArchiveDestPath%\IIS_VirtDir” /C /E /I /R /Y
ECHO Archiving %TEMP% Files from installation
XCOPY %TEMP%\*.log “%ArchiveDestPath%\TEMPDir” /C /I /R /Y
ECHO Archiving Migration Saved configuration files from installation
XCOPY “C:\Documents and Settings\All Users\Application Data\ISI Systems\Archive” “%ArchiveDestPath%\Migration Archived Files” /C /E /I /R /Y
ECHO Transferring Archived Files to localhost machine
NET USE \\DevShare\DevTesting\Charles\R7_Results /USER:DOMAIN\USERNAME PASSWORD ß Plug in your domain, user and password!
\\DevShare\DevTesting\Charles\robocopy.exe %ArchiveDestPath% “\\DevShare\DevTesting\Charles\R7_Results\%ArchiveDest%” /MIR
NET USE \\DevShare\DevTesting\Charles\R7_Results /DEL
rmdir /S /Q %ArchiveDestPath%
ECHO Forensics_Archive.bat – batch utility to copy installation results off of VM Image to Host machine share
ECHO Usage: Forensics_Archive “NewInstall\Iteration1_Dialogs”
ECHO.Note: Cannot have spaces in the directory name!!!!!
ECHO. Final results stored on
ECHO. \\DevShare\DevTesting\Charles\R7_Results\[Your Directory Name]
ECHO Written by Charles B. Hodge
:: Clean up variables and quit
Data Collection Comparison Method
As noted in a early posting – I have an affinity for the file/directory comparison tool “Beyond Compare”. I have found that once I have gathered the data – I can simply compare one directory against another. Very useful for comparing the known result set for an install with dialogs that I approved for TestCase #1 against the results set of a silent/unattended install that used the same selection criteria.
Hope this helps!