04/20/2009 by Serge Baumberger
The most recent Forrester Wave (Q3 2008) examined the six biggest developers of functional testing solutions. Based on the current offer and the strategy regarding the direction the corresponding solution is to take, HP was ranked number one and IBM came in second. It’s time to take a closer look at the current flagship products and compare them against each other.
In the red corner of the ring, representing IBM, please welcome Rational Quality Manager Standard Edition (version 1.002) and in the white corner, representing HP, please welcome Quality Center 10.0 (Enterprise Edition). In this bout, the current world champion Quality Center (QC) will be defending its title against the challenger Quality Manager (QM) with the fearsome left hook in the area of “look and feel” as well as in the key discipline of “range of functions.”
Appearance – Playfulness vs. Solidity
Both solutions are opened using a URL in the browser. Here, Quality Center (QC) 10 still depends on Active-X, which has been deactivated in many companies and for that reason usually has to be distributed as an additional package to the clients. Quality Manager (QM) doesn’t face this obstacle and therefore runs perfectly, including in Firefox.
Once accessed, QC and QM have very clearly laid out appearances. In QM, one always starts on a personal dashboard. This dashboard is very easy to configure and looks good, too. As described in the blog entitled HP Quality Center 10 from a Test Manager’s Perspective, the QC 10 also offers an integrated dashboard which has a similar range of functions as QM’s.
One navigates through both test suites using a vertical icon bar, which can be masked in the QC and arranged horizontally as an option in the QM. Overall, QM seems faster and more modern, more dynamic, and more interactive, whereby QC seems more plain and professional. QC can’t be changed much, with most of it controlled by means of lists and parameters and a little VBScript. On the other hand, QM can be totally reconfigured and adjusted to the nth degree in regard to a company’s internal workflows. If QM has more of a Facebook look and feel, then QC would be more like Xing; that’s why descriptions like playful versus grounded are apt.
Range of Functions
The getting-started hurdle for new testing solutions is high. In order to come close or beat the industry champion Quality Center, special moves are called for right in the first release. So what does Quality Manager include in its first release to take on QC?
It would be best for us to compare the individual mini-modules/functions that are needed in day-to-day work in a table to then pick out the corresponding winner:
Legend: 0 = Functionality/mini-module not available 1 = Functionality/mini-module available 2 = Functionality/mini-module better than the competitor's 3 = Functionality/mini-module is very good * = proved stability and scalability (QC) / modern web 2.0 interface without Active-X (QM)
In my opinion, the primary difference pertains to QM’s interactive test plan (planning), which QC lacks. The test plan should in no way be confused with the Quality Center’s test plan module. This is a bona fide test plan that is jointly generated through the distribution of work items (generally by the test manager). Individual items, such as requirements, test cases, or assembling corresponding test teams can be delegated to other QM users by means of work items. The test managers in charge can monitor progress at any time via the dashboard, making it ideal for offshore projects!
QM’s test environment management, which is not available in QC, is also pretty neat. If connected, test equipment can be provided with images at the click of a button and reserved accordingly for a test.
In contrast, Quality Center offers an integrated baselining tool and risk-based quality management, which we were unable to find in QM.
In a direct comparison, Quality Center is (still?) in the lead. However, with the JAZZ platform and the Open Commercial Development, Quality Manager has a speed advantage that needs to be taken into account. The current version of QM doesn’t have multi-project support yet, but it is expected to be available in April 2009 with version 1.01. On jazz.net, one can also check (“open” for that reason) what is currently being tinkered on. That’s one reason to look forward to fast release cycles. It remains to be seen whether customers want this type of speed from these testing solutions and will also import the new versions right away.
The QC versus QM match doesn’t end with a knockout. If one tallies up the points, the winner is QC with a score of 20 to 14 because it simply does many things in a better or niftier way than Quality Manager (importing test cases is supported in QC with a Word or Excel plug-in; in QM, the upload is done without support in the XML format). But even here, QM will probably catch up quickly. This is also necessary since QM’s release 1 deals with a new solution which in everyday work still has “teething pains” (e.g., test cases can’t be copied and saving attachments is sometimes impossible).
Ultimately, it’s worth noting whether one’s company already relies on IBM Rational products and the JAZZ platform in general, because combining development, requirements engineering, and testing on the same platform has a lot of cost-saving potential. To top it off, choosing a functional testing solution is always a price-related issue.