Peter HelfensteinPeter Helfenstein E2E SAP Testing – Halfway Under Control?

08/05/2008 by Peter Helfenstein

In many blogs, we have discussed the deficiencies in SAP testing. Changes to an existing system are the order of the day. Again and again, errors are corrected, customizing and programs must be adapted, new functions are rolled out. Also, technical changes as hot fixes and service packs must be installed, as well as upgrades. In view of the large number of changes, problems with testing is not want you really want. But most SAP customers have come to terms with it, solving retroactively the problems involved with rolling out badly or incompletely tested changes – amidst the applause of the business. This is quite an effort, but still considered a more sensible approach than testing the whole software system.

Everyone will agree that this is not the approach we want. Fortunately, specialized consulting companies such as FocusFrame as well as manufacturers of testing software such as HPPanaya, IntelliCorp, IBIS and above all SAP have taken this to heart. However, we are still far from a real end-to-end (E2E) testing or quality process starting right from the business requirements level, but at least you can start a continuous E2E testing process covering all changes as soon as the transport of a SAP change request is due. And this is already a great benefit.

What Does This Mean in Detail?

SAP E2E testing process and tools

  • The first step is to analyze which SAP transactions and programs are affected by the change, on the basis of the changes due for transport and of the system or a model of it. This step is mostly referred to as Change Impact Testing. This task is supported by specialized products like HP Change Impact Testing (CIT) for SAP, SAP Test Automation and Optimization (SAP TAO) as well as IntelliCorp LiveCompare. An exciting alternative is Panaya Inc.´s pure SaaS solution. Using this service, the SAP user does not even have to install any software.
  • Once the involved transactions are known, it must be found out which testing scenarios are necessary. For this step, too, there is reasonable technical support now. As a result, the testing scenarios are identified and together with the underlying test script made available to the test management tool. This is accomplished by using SAP Test Automization and Optimization in combination with SAP Quality Center by HP.
  • In spite of above system support and test szenario reduction, the testing scenarios should – due to their large number – pass another filter. This step is widely referred to as Risk Based Testing. The point here is to reduce the number of tests to be executed to the really relevant ones. This step is supported by SAP TAO and, particularly, by SAP Quality Center by HP.
  • Then, the test management tool executes the filtered tests automatically. The results are evaluated and, if necessary, the defect management process is started and monitored. This task is assumed by SAP Quality Center by HP in combination with HP Quick Test Pro.

From Transport Packet-to-End-Testing up to real E2E Testing
All this sounds quite simple and logical. In practice, however, it is quite a challenge to set up and operate such an environment. As a methodologist, however, you will consider this „reactive“ approach still suboptimal as the above „E2E“ quality process, instead of starting from the beginning, meaning the requirements management, is initiated not before the changes have been executed and ready for transport. Information about the effects of changes is known to be useful on starting the projects from the project portfolio management, supporting the business analyst as well as the developers and the test team. SAP has become aware of this and is working on a solution using the new Business Process Change Analyzer (SAP BPCA) to enable real E2E quality management.

Implementing SAP E2E testing correctly will change the poor SAP E2E quality mangement into a model for other commercial software solutions addressing quality management. The benefit of a consistent, comprehensive testing for SAP landscapes can hardly be overestimated. The reduced number of system breakdowns after the weekly roll-out of changes justifies the investment in this area – as every head of a SAP Competence Center will readily confirm. The somewhat less spectacular quality improvement along with the cost and risk reduction reached in the medium term, will provide a considerable profit by introducing a SAP E2E testing project in the context of an application lifecycle management initiative.

Sphere: Related Content

No TweetBacks yet. (Be the first to Tweet this post)

6 Responses

  • 1
    Brian Anderson:

    Dear Mr. Helfenstein,

    Thank you for the blog on E2E SAP Testing. Your article is very similar to a presentation I made on SAP Test Automation and Business Process Validation at this year’s North American Sapphire/ASUG event in Orlando to over 200 attendees.

    Worksoft provides an automated testing solution focused on SAP, but also supporting other platforms, including HTML, Java, .Net, VB, SOA, and even mainframes. Worksoft is also an IntelliCorp partner and we resell Live Compare as well as other IntelliCorp products re-branded as Live Model and Live Data.

    Worksoft’s testing solution is unique in that it is completely script free. Test are defined and executed using a powerful object-action model. This means that business analysts and other SAP functional users (the folks who really know how SAP should operate in their implementation) can easily define and execute tests through Worksoft’s menu driven interface. This object-action model also means that tests can be easily and automatically updated base on the changes you describe to SAP or any other application under test.

    Please let me know if you would be interested in more information on Worksoft’s solution. We have a white paper I can send based on the Business Process Validation topic I mentioned above and we would love to be included on your list of Functional Testing solutions.

    Thanks again for the great blog. Now that I’ve found it, I will check back often.

    Brian Anderson
    Director of Product Management
    Worksoft, Inc.
    banderson@worksoft.com
    (214) 239-0409

  • 2
    Amit Bendov:

    If I may add, it is important to note that there’s also substantial cost and time saving created.
    For example, we find that most customers we work with have overestimated the testing budget for upgrades and support packs, since they do not know what to test. The returns from smarter analysis that will tell what needs to be tested, and equally important, what not to test are fantastic.

  • 3
    Peter Helfenstein:

    Brian, we are glad to look more detailed into Worksoft’s testing solution and to add it to our SAP Quality Management tools miniguide. Maybe it is worth its own blog post soon. I will contact you via email.
    Welcome to the beteo SAP ALM blog community.
    Peter

  • 4
    Peter Helfenstein:

    Amit,
    many thanks for your feedback. You are so right. For the SAP IT organization cost and time savings through SAP E2E testing are substantial. 30%-40% productivity gains at SAP customers using services or software like Panaya when it comes to upgrades or major changes are typically reported. This is very impressive.

    From a business point of view though the increased quality reducing the risk of application failures after regular change roll-outs is to me is equally if not more important.

    Peter

  • 5
    HP Quality Center 10 from a Test Manager’s Perspective /// beteo:

    […] Along with version control comes baselining, which is intended to answer the question above. Using the new “Management” module that replaces the “Releases” introduced in 9.x one gets to the baseline function via the “Libraries” tab. This enables one to obtain a summary of a complete testing release and retrieve it if necessary. In this way, a test set can be pinned to a baseline in the test lab. In other words, as of now, the manual copying of entire trees into the test plan module is a thing of the past. At last, test managers will be able to properly organize software that has multiple parallel releases (in production, current release, future release). […]

  • 6
    HP Quality Center 10 from a Test Manager’s Perspective « Software Testing and QA:

    […] Along with version control comes baselining, which is intended to answer the question above. Using the new “Management” module that replaces the “Releases” introduced in 9.x one gets to the baseline function via the “Libraries” tab. This enables one to obtain a summary of a complete testing release and retrieve it if necessary. In this way, a test set can be pinned to a baseline in the test lab. In other words, as of now, the manual copying of entire trees into the test plan module is a thing of the past. At last, test managers will be able to properly organize software that has multiple parallel releases (in production, current release, future release). […]