The full set of unit and integration test scripts are executed daily as part of an automated build and test process. The success or failure of the test scripts are monitored closely by developers and the QA team. Upon certification of the build and preparation of the QA environment, the successful builds are deployed on the QA servers.
Once the new build is on the QA server the product undergoes a series of smoke tests. The approach for smoke testing varies from project to project, but at a minimum the set of unit and integration test scripts are executed to ensure a successful deployment and correct operation of core application features. In addition, core functionality as documented in release notes is specifically tested to ensure proper behavior.
For most organizations, QA simply means functional testing. We take it much further. Functional testing, regression testing and system testing are performed through out the development cycle.
We know what it takes to make software successful, so we extend the testing to check for its RAS (Reliability, Availability, and Scalability) properties. And when building n-tier applications, we know that this needs to be done at each stage.
Automated build and deploy scripts test daily builds on designated machines in the QA Lab. The lab has been equipped with machines with almost all flavors of Operating Systems, Databases, Application Servers, etc. We keep ghosted images of these systems for quick redeployment as the need arises.
Performance testing also requires clustered environments which we maintain in the lab. We use load testing tools against these clustered environments to test for scalability and to allow the client to do some strategic capacity planning as required.
Upon certification of a release for production by the QA team, the product is deployed. Deployment is often a forgotten item. Software actually needs to be installed at the user’s site for it to be useful.
Clean machine tests are always performed to make sure that the software will work in an environment without the normal debugging tools. Testing upgrade installations is, in many ways, even more important since the software is now going to be installed on a production system. Any mistake that can cause the loss of data is disastrous.
Many companies have a dedicated set of staging servers that closely match the production configuration. In this environment the product is deployed and the QA team performs an extensive array of regression tests, unit test, and manual tests to ensure stability.
For new or major releases the QA team will also perform performance testing, which includes stress and load testing of the product. One of the common tests is an extended load test that executes for several days to ensure that the product will run successfully in a live environment. After successful completion of performance testing and client acceptance of the product, the product is deployed on the production servers, where basic regression testing occurs prior to bringing the release live.