Application Performance Explorer Reference

Application Performance Explorer (APE) tests the run-time performance and resource load characteristics of distributed application design scenarios you are considering. With APE, you can run automated "what-if" tests to demonstrate the resource consumption implications of architectural choices you might make.

Distributed applications can use various logical and physical design configurations. It's often difficult to determine how to optimize your application design to fit the workload characteristics of your run-time environment. APE provides a way to test your application design assumptions, and therefore determine which design options work best in certain scenarios and identify tradeoffs between design options.

The following sections provide information to acquaint you with Application Performance Explorer.

Section | Description
Application Performance Explorer Concepts | Describes how APE can benefit your application designs and how to use it to performance tune your distributed application.
Application Performance Explorer Common Tasks | Provides links to step-by-step topics on common APE tasks.
Application Performance Explorer User Interface Reference | Describes the user dialog boxes and interface elements.
Application Performance Explorer Error Messages | Provides links to topics explaining APE error messages you may encounter, along with workarounds.


Application Performance Explorer Concepts

Application Performance Explorer (APE) models your application design and tests the expected performance and interactions of a distributed architecture. You can save favorite design architectures and use them as the basis for new configurations and performance tests.

With APE, you can understand the consequences of certain design choices and test design alternatives as you tune your application's performance. You can automate the tests, scheduling them for certain clock times to experience peak network traffic or other throughput parameters. Additionally, you can modify APE's source code for customized performance testing.

The following sections provide information to acquaint you with Application Performance Explorer.

Section | Description
Why Use Application Performance Explorer? | Describes how APE can help improve your application designs by testing various architectural and deployment configurations.
Understanding How APE Works | Describes the APE architecture and how it works.
Using Application Performance Explorer | Explains how to install APE and run a simple test.
Defining APE Profiles | Describes the profile as a specific — yet modifiable — architectural configuration that forms the basis for an APE performance test.
Performance Tuning Your Application | Introduces some good application design ideas and tests the resulting performance. Each test comparison includes the setup values and the generated data.
Customizing Application Performance Explorer | Explains how to customize the APE component source code to more closely model your exact test requirements. Additionally, suggests using APE source code for a new project.


Why Use Application Performance Explorer?

Application Performance Explorer (APE) helps with the design process by setting up test cases with various logical and physical deployment configurations, and measuring performance under run-time workload conditions you define. Using APE to define and save profiles that prototype your design choices, you can adjust and evaluate those choices more easily than has traditionally been possible.

One of the key assumptions in designing modern multi-tier, client/server, distributed applications is that components are valuable. Using components you can break up a big problem into smaller, more manageable pieces, often with the benefit of reusing work from other developers. In addition, in component-based applications, you can use valuable design options such as asynchronous execution, remote resources, and differing logical and physical deployment configurations.

It's not unusual for a developer to have very little experience with multi-tier, distributed component architectures. Unfortunately, the design assumptions that often lead to a successful workstation application are often tragically wrong for a component-based, distributed enterprise application. In the classic desktop or mainframe programming scenario, resources are fully dedicated to the application's demands. In distributed applications, this idea of on-demand allocation of shared server resources is actually an expensive and damaging notion that developers should not carry from their desktop-centric application model.

For example, while it's true that the location transparency of DCOM allows an application to call remote objects as if they were local, this model is not efficient as the workload increases. The overhead of creating the objects and the associated resource consumption required to satisfy dozens or hundreds of simultaneous users eventually consumes a machine to the point of thrashing. Instead, design strategies that incorporate service queuing and object pooling can provide higher performance in distributed application architectures.

For More Information You can find guidelines for designing enterprise applications in Defining Application Logic.

In order to understand the performance opportunities and design consequences of your distributed applications, you can use Application Performance Explorer to create real run-time scenarios, stressing all of the application's infrastructure in accordance with adjustable settings.

APE is configurable to show the performance implications of client loads, network bandwidth, machine boundaries, component instancing and reuse, queuing, DCOM business service processes, data access and transfer methods, server capacity, and various physical deployment options.

Design and performance testing options include modeling various Microsoft BackOffice technologies such as Microsoft Transaction Server (MTS) and Microsoft SQL Server.

Application Performance Explorer can help you understand and evaluate different design strategies and discover the optimum architectural model for your application by providing:

  • Design performance evaluation.

  • Easily configured and storable design scenarios.

  • Reference source code examples demonstrating recommended design technologies.

In addition, APE is itself an example of a well-designed distributed application. The APE source code is commented, and is structured to serve as a template for component-based applications. This source code has been designed so that you can use it as the starting point for your own distributed multi-tier application.

For More Information The source code for the Application Performance Explorer is in the ..\Program Files\Microsoft Visual Studio\Common\Tools\Ape\Source subdirectory of your Visual Studio installation.


Understanding How APE Works

The Application Performance Explorer models application design by using three tiers. These are:

  • Clients

  • Servers

  • Databases

These tiers and the associated processes and services are summarized in the following illustration.

Clients

When you click Start on the APE Manager interface, at least one client process is created. Each client then creates services by using one of the following techniques:

  • Direct instantiation.

  • Using the Pool Manager (the pooling option).

  • Using the Job Manager (the queuing and pooling option).

Additionally, a client can directly access the Jet database, simulating a typical client/server application architecture.

Client processes can pass data to the service objects with each call. In the asynchronous model, the data is passed via the job manager and a worker with each service request. In the synchronous model, data is passed only via a worker.

Both the Pool Manager and Job Manager create workers that in turn create services. When the Retain Service References option is checked, the workers keep a reference to service objects as they are used. The worker then only has to instantiate a service object the first time. After that first time, the worker uses its stored reference to the service to accomplish tasks.

After completing its work, the client returns its call to the APE Manager via the Logger. This logging function accepts information from many sources, including:

  • Client events

  • Job Manager events

  • Pool Manager events

  • Worker events

  • MTS events

  • Database events

  • Expediter events

  • Job summary information

Servers

APE supplies a set of worker services that emulate typical CPU and database access tasks. These workers are instantiated either directly by the client or via the Pool Manager (pooling) or Job Manager (queuing and pooling) services. In a manner similar to the clients, the workers return their results to APE Manager via the Logger.

For More Information For more information on configuring the workers, defining the resource consumption characteristics of the services, and an explanation of how queuing and pooling affect your APE test, see Defining a Server.

Databases

To provide a variety of database access modeling, APE supports both Jet and SQL Server databases. You can model your data access scenarios using several types of data access methods, including:

  • ActiveX Data Objects (ADO)

  • Remote Data Objects (RDO)

  • Data Access Objects (DAO)

  • ODBC API

For modeling typical client/server applications, the data access work flows directly between the client and the Jet database. For modeling modern distributed applications, the data access work flow is between the worker and either the Jet or SQL Server database.

The data access connection and actual database events use real queries and tables to provide accurate data access modeling. You can change the database to create the exact data access events required by your application.

For More Information For more information on how data access technology affects the design and performance of your application, see Choosing the Right Data Access Technology. For more information on how to change APE's database tables and queries, see Defining a Database.


Using Application Performance Explorer

The following sections provide information on installing and using the Application Performance Explorer.

Section | Description
Installing Application Performance Explorer | Explains how to install APE.
Resolving APE Problems | Explains how to fix common APE problems and achieve reliable.
Running a Simple APE Test | Explains how to run a simple APE test.
Getting Reliable Results | Discusses certain test strategies for achieving good test results.


Running a Simple APE Test

The basic steps to use Application Performance Explorer are:

  • Selecting a profile.

  • Configuring the component objects.

  • Running and evaluating the test results.

Selecting a Profile

With Application Performance Explorer you can define a variety of parameters to determine their overall impact on your application's performance. Each of the profiles uses the same set of component object types, with user settings to control various aspects of their functionality.

APE profiles are divided into two categories: asynchronous and synchronous. Each profile contains preconfigured settings for each of the component characteristics, quantities, connection options, workers, services, and clients. All of these settings collectively form the profile of the application design scenario to be modeled. You can also create your own profile by selecting an existing profile, individually configuring the component services, and then saving the profile with a custom name.

Configuring the Component Objects

You configure each type of object by clicking any of the five large buttons on the APE Manager user interface.

The buttons display dialog boxes that specify the number of objects to be created, data passing options, timing requirements, callback options, logging options, and other variations in the construction and use of the objects. Configuring the component objects provides great flexibility for modeling your enterprise application. You can even specify your own component service objects.

Running and Evaluating the Test Results

Once the test configuration is complete, you click Start to run the APE test. As the test runs, certain test events such as creating the clients and service objects appear in the results window. When the test is complete, you can view the overall test performance in the results window. As you become familiar with how Application Performance Explorer works and what it can do, you might want to store the detailed test information by configuring the log file to save selected test events.

Running a Simple APE Test

In the procedure below, you will select a typical APE profile, make a few simple modifications, run a performance test, and examine the output.

To run a simple APE test

  1. From the Start menu, click Programs, point to Microsoft Visual Studio 6.0, Enterprise Features, and then click Application Performance Explorer.

    The main Application Performance Explorer window appears.

  2. From the Profile list, select a profile. For this test run, select "Typical performance, asynchronous (CPU, Job)."

  3. Click the large Client Process button.

    The Client Options dialog box appears.

  4. On the General tab, set the Number of Calls to 50.

    This shortens the duration of the test to the time it takes to make 50 client calls (instead of the default 100 calls).

  5. Click the Concurrency tab.

  6. On the Concurrency tab, set the Client Processes per Machine to 4.

    This causes the test to allocate four concurrent client processes on the single APE Manager machine. More than one client process is realistic, since a user might be expected to simultaneously run at least four applications (such as your enterprise application, word processing, scheduling, and e-mail).

  7. Click the Start button to begin the test.

  8. Wait for the test to finish or click the Stop button.

  9. View the test results in the results pane at the bottom of the screen.

    The results window shows that all requested clients were successfully created for a test run total of 200 calls (50 calls for each of four client processes). The Elapsed Time, Client Calls Per Second, and Overall Calls Per Second are also indicated.

  10. Click Clear to clear the results window.


Getting Reliable Results

APE's test results can be affected by other application and network events. The following list will help you achieve the most reliable test results.

  • Do not run any other applications on the machines that APE is using for its tests. Other applications can randomly consume memory and CPU cycles, making it hard to achieve consistent results with APE.

  • For the fastest possible performance results, use a private network between all APE machines, or at least try to run APE tests at times when network traffic is minimized. On the other hand, running your APE test during typical network traffic conditions may present a more realistic test scenario.

  • Run multiple tests, discarding the results of the first run that often has unique setup costs. Long tests (using hundreds or thousands of calls) will produce more accurate average numbers.

  • Test performance can vary at different times of the day or with different hardware configurations. To guarantee that tests are repeated accurately, you can use named profiles to save and recall test settings. If further control is needed, custom test scripts can be easily written in VB (or a VBA application like Microsoft Excel) to control the tests through APE's COM programming model.

  • APE's performance tests can be customized to match very specific application scenarios by modifying the APE source code, recompiling the changed components, and redeploying and reregistering the modified components to the appropriate machines. The two components that emulate business object behavior and will likely be the most interesting to customize are AeServic and AeMtsSvc.

    Note The Visual Basic source code and project files for all of APE's components are located in ..\Program Files\Microsoft Visual Studio\Common\Tools\Ape\Source.

  • After running a test, you can verify the successful completion of write queries and MTS transactions by looking for changing values in the ACCOUNT table of the APETEST database. Each Simple Write query adds one dollar to an account selected randomly from a range of one through 1000. Each Simple Transaction query moves one dollar between two accounts selected randomly from a range of one through 1000.

For More Information For more information on monitoring MTS transactions, see Monitoring MTS Transactions.

  • If your client configuration does not send or return data, all the client is doing is submitting the call. As your test is running, the client's Calls Returned count merely represents how many calls were accepted. If your test is also using queuing, the Calls Returned count will quickly reach the designated test amount — even though the queue may still be handling various CPU or database tasks. This is expected behavior. To actively involve the client in a relationship with the queuing activity, you must configure the test run to Return Data to Client (on the Return Data tab of the Client Options dialog box).


Defining APE Profiles

Enterprise applications typically partition functionality into separate components that are distributed on the network in a variety of physical deployment configurations. Application Performance Explorer uses profiles to quickly model such architectural options and then measure the run-time performance under different network and deployment options.

The APE Manager interface contains five buttons that display dialog boxes for changing the various test configurations. These settings collectively become the "profile" of the scenario to be modeled.

The following sections explain some of the important profile options you can use to configure your APE test run.

Section | Description
Defining a Client | Describes some of the important client configuration options.
Defining a Connection to a Server | Explains the available connection options for remoting all of APE's objects.
Defining a Server | Discusses some important worker and service configuration options.
Defining a Connection to a Database | Describes the data access choices for connecting to a database.
Defining a Database | Describes the two database choices: Jet and SQL Server.
Defining a Log | Explains how to configure the APE logging process.


Defining a Client

The client machine is the single workstation on your desktop. The major client parameters are the number of client processes, the number of calls that each client makes, and (optionally) the size of the data to be passed. Varying the number of clients is useful for loading the system and for examining the advantages of a queue manager with extremely large numbers of clients. Note that it is often impractical to set up 100 or more workstations to test the effect of a large number of users connected to the system. It is, however, more practical to set up 10 clients and increase the call frequency to simulate high-traffic conditions.

The number of calls, pooling, and service binding options are useful for examining the effect of component persistence. As the number of calls increases, you'll want to take advantage of component persistence (using the Pool Manager) to handle the workload demand. This is because when a component stays in memory between calls, the system spends less time creating and destroying objects and can support a much higher volume of work. Early service binding will improve the speed of your application. Also, varying the size and type of the data can be useful in discovering major performance problems with network throughput.

If your test requires remote clients, remember that each remote machine must be properly configured with APE components. For step-by-step instructions on installing APE components on remote machines, see "Configuring Access Permissions Using DCOMCNFG" in the topic Installing Application Performance Explorer.


Defining a Connection to a Server

The APE Manager and all of its component services can run on either a single Windows 95 or Windows NT 4.0 (or later) machine. If your APE test uses remote components, those workers and services must be located on a Windows NT machine in order to support distributed COM (DCOM).

The connection choice configures the remoting option for all of the object classes used by the Application Performance Explorer. You can choose from either DCOM or Remote Automation.


Defining a Server

Server definition is a significant part of APE modeling. This is where you choose pooling and queuing, configure the workers, and define the resource consumption characteristics of the services.

Pooling is an important design strategy because it quickly recycles objects that have already been created. Queuing is also important because with queuing your application handle increased workloads while balancing resource consumption. You can select pooling only, or both queuing and pooling together (for asynchronous models only).

You can configure the worker options using the following four parameters.

  • Number of Workers As previously indicated, this is only relevant for the Asynchronous Queue model. It determines the number of worker objects that are "pre-instantiated" by the queue.

  • Early Bind Services Object controllers can bind to object interfaces in one of two ways: late binding using the IDispatch mechanism or early binding using a vTable. In general, object variables that are dimensioned "As Object" are late bound: each method or property address is looked up at run time. Object variables that can take advantage of type libraries and be dimensioned as specific object types (for instance, "As Worksheet") at compile time are said to be early bound. Early binding offers a number of advantages, the most important of which is substantially better performance.

  • Retain Service References In many object models, an object is created, used, and destroyed each time it is called. The Retain Service References option allows the worker to keep the service object in memory for repeated calls against the same object. This is most useful in the queue manager model. In the Direct model, this option only has an effect if the number of calls (set on the Clients tab) is greater than one.

  • Preload Services Workers are created in the Queue Manager model before any Client requests are processed. This option forces the Worker, when instantiated, to load its corresponding Service. Setting this option eliminates this overhead from the performance log.

The CPU task option simulates real work by specifying task and sleep duration. You can investigate CPU availability by adjusting the CPU task workload on server machines.

Regardless of the work simulation determined by these settings, the only "real" work performed by APE's built-in service components is to return the data requested by the client. For more accurate measurement, you can substitute your own object method as a service. To do this, you specify a string of the form "Library.Class.Method", where Library is typically the name of your object server, Class corresponds to one of its publicly exposed object classes, and Method is any method implemented by the Class that requires no parameters.


Defining a Connection to a Database

APE provides the following four data access methods.

  • ActiveX Data Objects (ADO)

  • Remote Data Objects (RDO)

  • Data Access Objects (DAO)

  • ODBC API

ADO is a universal data access method that provides a consistent interface to any OLE DB data provider. ADO is the recommended data access solution for modeling (and eventually building) new applications.

RDO is specifically designed to access remote ODBC relational data sources. If your APE test must model an existing RDO application, this choice will work for you. On the other hand, if you are testing design options for a new application, you should consider using ADO.

DAO is an older data access technology and is more difficult to implement than either ADO or RDO. If your application design has a specific requirement for DAO, then the DAO connection option will work for you. If you are using APE to model a new application design, you should consider the design, coding, and performance benefits provided by ADO.

The ODBC API is a widely available interface for relational data. While it can be difficult to write ODBC code, it is fast and provides a common way to access different relational database engines. If your APE test must model an existing ODBC data access scenario, then this choice will work for you. If you are designing a new application, you should consider using ADO.

For More Information For more information on data access technologies and a discussion of which methods you should use for your application, see Choosing the Right Data Access Technology.


Defining a Database

APE has the following two database options to select from.

  • Microsoft Jet engine

  • Microsoft® SQL Server™

Your choice of which database to test with depends on the data access requirements of your application. While the Jet engine is behind what are sometimes referred to as a personal databases, it can handle a moderate number of simultaneous users. If your application is small and does not require the benefits of a large-scale database, you might want to test (and ultimately implement) your application using Jet.

If you are modeling an enterprise application that requires a robust, high-performance database, you should test using SQL Server. With the SQL Server test option, you can actually modify the APETEST database tables and queries to represent your application's final data access requirements.

The default query statements that APE uses are stored in the ApeQuery table in the Apetest.mdb database. You can use Microsoft Access to change these queries or add some of your own. One useful idea is to add a simple query such as exec myproc and then create a stored procedure in SQL Server with the same name. In this way, you can execute an APE test that realistically runs your own custom stored procedure.

Tip If you insert your own query into the ApeQuery table, be sure to use the Applications Service Options dialog box to select the query.

For More Information For more information on installing the APETEST database for SQL Server, see Installing Application Performance Explorer.


Defining a Log

Each APE component independently provides the option of displaying status forms and logging event information. The logging process simply builds a *.log text file to record the step-by-step progress of your test and the actions of each of the APE components.

Because of the performance impact of logging to disk, the log can be maintained in memory and written either when the scenario is completed or when a predefined maximum log size is reached.

Note APE uses the Windows Notepad program to view the log. In some cases, however, the log file can grow quite large and exceed the capacity of Notepad. If you encounter a message indicating that the file is too large, manually open the file with another viewing program (such as Wordpad), or rerun the tests with less logging information or for a shorter duration.


Performance Tuning Your Application

The key performance issue for distributed applications is not tuning raw code execution speed, but rather determining the optimum physical packaging of components, the best resource sharing model, and the best network topology for an effective distributed solution. With APE you can model various deployment options by putting all of the services on the same machine, putting the business processes and the client services together on one machine with the data access on another machine, or by defining many other physical deployment options. Additionally, you can model the performance and scalability benefits of both queuing and object pooling.

What APE is really measuring isn't so much the work that your application will do (which APE can only simulate), but rather the capacities and throughput performance of the distributed infrastructure your new application will use. Because the performance of this infrastructure is often a critical factor in the success of any enterprise application, the use of APE early in the design phase of a new application can significantly improve its chances of success.

To understand the significance of the various design factors APE can benchmark, you need to learn how remote components can be deployed either synchronously or asynchronously. Each of these approaches has its advantages and disadvantages, and different parts of your application can use different models, depending on the needs of the tasks being implemented.

The following topics introduce the common performance issues of a distributed application, describe the basic APE remote deployment models, and explain certain performance tuning concepts that are important to your enterprise application.

Section | Description
Understanding How Distributed Components Affect Performance | Describes how component size and location can affect the general performance of your application.
Understanding APE's Remote Deployment Model | Describes the three APE deployment models: synchronous, asynchronous, and queued.
Common APE Performance Comparisons | Examines the test results of three basic APE profiles.

For More Information For more information on performance tuning from a data access perspective, see Data Access Performance Tuning.


Understanding How Distributed Components Affect Performance

Distributed applications partition functionality into separate components that can be deployed on the network in a wide variety of physical configurations. The key performance issue for these distributed components is not tuning raw code execution, but determining the optimum network topology for a distributed solution.

The first and most important question that you must answer in considering how best to take advantage of the potential for distributed computing — and whether a specific application task should in fact be remotely executed — is this: Is the overhead associated with the remote component worth the benefit?

If you choose distributed component architecture for your enterprise application, then you must determine the appropriate component granularity and physical deployment. Granularity has to do with how objects and logical services are sized and grouped into physical components. Deployment considers where each component should be located in order to achieve the required performance.

You can directly examine the performance effects of component granularity and deployment by exploring alternative design questions such as:

  • Is it more efficient to package services in a single, large component or to break up functionality among many smaller components?

  • Is it better to deploy a particular computation-intensive task on a remote server in order to take advantage of superior computing resources, or on the local desktop machine to minimize network traffic?

  • What is the effect upon my design when adding more clients?

  • Where are the bottlenecks of my solution during the worst network conditions?

  • What is the effect of object pooling and recycling?

  • Would queuing help when requests to a back-end server exceed its capacity?

With Application Performance Explorer you can answer these design performance questions by quickly setting up test cases with differing granularity and deployment options.

At a detailed design level, the question of whether or not to use remote component deployment for a particular task requires consideration of the following general design parameters.

Design parameter | Consideration
Task size | How big is the task being considered for remoting?
Request frequency | How often does the task need to be executed? This includes requests from all users, not just a single desktop machine.
Network bandwidth | What is the raw speed of the network cable, network interface card(s), network protocol, RPC, and drivers, as well as the collision/retry overhead caused by network traffic?
Data transfer requirements | How much data needs to be transferred to and from the task? In some cases this value may be negative because the location of the task on a remote server may actually reduce data transfer requirements to the desktop.
Desktop capacity | What is the raw bandwidth of the desktop system's CPU(s), memory, disk I/O, and other resources and devices that are needed to process the task?
Remote capacity | What is the raw bandwidth of the remote system's CPU(s), memory, disk I/O, and other resources and devices that are needed to process the task?
Desktop opportunity cost | What could the desktop machine be doing if it were not running the task?
Server opportunity cost | What could the server being doing if it were not running the task?
Desktop resource burden | What resource burdens do the operating system and other concurrent applications impose on the desktop system?
Shared resource burden | What resource burdens do the operating system and other concurrent applications impose on the shared system?
Application design and implementation | How efficiently are these factors considered in the design and implementation of an application?
Centralized administration | What is the benefit of administering or updating the task from a centralized location rather than each desktop?

From the design parameters listed above, it should be obvious that there is no simple quantitative answer to distributed component deployment. Because each target installation has a unique resource infrastructure that is not duplicated anywhere else, every application needs a different deployment design answer. No tool can answer all of these questions, but Application Performance Explorer can provide meaningful insights and measurable answers to many of these deployment and performance questions.


Understanding APE's Remote Deployment Model

APE models remote deployment using two separate components working closely together: a worker and a service. This is good component-based design, separating the execution context (worker) from the actual computational routines or business services (service). Since remote components are most efficiently packaged as in-process servers, they need a process space on the remote machine in which to execute. The worker component provides this process space and execution thread, while the service encapsulates the application-specific functionality.

As with all remote components, the resource capacity of the remote machine often limits the amount of work that can be done. In order to use remote components as part of a high-performance enterprise application, you must understand APE's three basic remote deployment models and how they affect your application's performance. These are:

  • Synchronous Communication Model

  • Asynchronous Communication Model

  • Queued Objects Communication Model with Callbacks

The following sections will acquaint you with the important remote deployment design concepts that you can configure and test using Application Performance Explorer.

Synchronous Communication Model

The Application Performance Explorer implements synchronous connections as a direct request from a client application to an object that lives on a remote network server. Because the connection is synchronous, the client application waits for the task to be completed and is blocked until the server returns.

In step 1 of the preceding illustration, the client creates a worker, which in turn creates a service that performs the work. When the service is finished, both the worker and service are destroyed. In step 2, additional clients create additional workers and services, never sharing them with other clients.

Direct object instantiation and executing repeated calls on a method is most useful for exploring network latency. Changing calling parameters and return value sizes can help determine network throughput, which can be useful in deciding which components should be run across a network on remote computers and which should remain together on a single machine.

One of the limitations of the synchronous approach is the overhead involved in starting the worker and service objects. When a client is done with a worker, both the worker and service are destroyed.

Asynchronous Communication Model

Without queuing, when a burst of requests comes in the object pool starts growing, the objects can't recycle fast enough, and the new object allocations rapidly degrade the server into a thrashing state. Without queuing, you must have sufficient memory on the server to deal with the absolute peak load, which is often many times more than the hardware needed for normal processing loads.

The asynchronous communication model keeps workers "alive" and reduces the overhead of initializing them. There are a number of approaches to this, including pool manager and queue manager scenarios. APE implements a queue manager as shown below.

In the simplest case, in step 1 the queue manager creates a given number of workers, all of them initially having a "not busy" status. In step 2, when a client request comes in, the queue manager allocates a worker and marks it as "busy." In step 3, the queue manager uses the worker to perform the service until it is completed, and changes its status back to "not busy." In step 4, the queue manager accepts all requests and allocates workers on a first-come, first-served basis.

Because the queue manager maintains its queue internally, client requests are never refused. If all pre-allocated workers are busy, the queue manager simply waits until one becomes available and assigns it to the next waiting client in turn. In distributed solutions, such a queue can be used to keep workers busy. This results in servers running at their optimum performance level — that is, fully loaded.

The choice to communicate asynchronously has another particular advantage: if the receiver is not running at the same time as the server, synchronous communication attempts will always fail; asynchronous communications can still succeed.

The performance value of queuing is that it allows your hardware to work at capacity without mismanagement or inefficient use. A side benefit of queuing is that, during peak workloads, the application's throughput degrades gracefully. Queuing is an important design concept required for high-performance, robust applications.

It's important to notice that this asynchronous deployment model has no provision for either returning status information to the client or logging transaction information for administrative purposes.

Note Although the current implementation of APE does not directly use Microsoft Message Queue Server (MSMQ) during performance test runs, APE does have a configurable Job Manager queue that can be adjusted to realistically model your application as if it were actually using MSMQ.

For More Information For more information on why queuing is an important design concept, see Performance Value Of Service Queuing with MSMQ. Microsoft Message Queue Server (MSMQ) is available in the Windows NT Service Pack 4. For more information on installing and using Microsoft Message Queue Server, see information about Windows NT Server at the Microsoft Web site at http://www.microsoft.com/ntserver/.

Queued Objects Communication Model with Callbacks

The queued objects model with callbacks has provision for either returning status information to the client or logging transaction information for administrative purposes.

In general, return information can be conveyed in one of two ways: either synchronously through a return value, or asynchronously through an indirect notification mechanism. The synchronous (direct) method results in the client program being blocked waiting for the return to complete. For asynchronous operations, a separate notification mechanism is required. Callbacks, available in Visual Basic, provide such a mechanism. Callbacks allow the client to continue processing and be notified asynchronously when the server side has something relevant to report.

To use callbacks in the queue model, an additional component, the expediter, is used to queue return information back to the client.

In operation, the client implements an internal Callback class with a predefined method. In step 1, the client then creates an instance of that callback object and passes a pointer to it to the queue manager, along with its work request. In step 2, the queue manager in turn calls the worker, passing the callback object pointer. In step 3, when the task is completed, the worker returns information to the queuing manager, once again passing along the pointer to the callback object. In step 4, the expediter calls back asynchronously into the client object.

Notice that this diagram also illustrates the queue manager, worker, and expediter all feeding into a logger.


Common APE Performance Comparisons

APE provides many built-in profiles for testing the remote execution, scheduling, and managing of objects both synchronously and asynchronously. Using preconfigured profiles, you can easily run some basic APE tests comparing the performance of some typical architectural designs.

While there are many test profiles to choose from, the three examples listed in the following table will acquaint you with some typical APE profile comparisons. You should read through each of the examples to see how the profile design choices affect the performance.

Note After running the following profile examples, you should experiment with workload increases such as doubling the number of calls or the number of clients. Such experimentation will show how the queuing and pooling design options allow your application to scale gracefully as the workload increases.

Profile name | Description
Typical performance, synchronous (DB, direct, visible) | Defines a performance test using a synchronous connection that causes the client to wait.
Typical performance, synchronous (DB, Pool, visible) | Defines a performance test that uses a synchronous connection with object pooling.
Typical performance, asynchronous (DB, Job, visible) | Defines a performance test for a scalable, robust application using asynchronous connections, queuing, and object pooling.

APE Comparison #1: Typical Performance, Synchronous (DB, direct, visible)

The Application Performance Explorer implements synchronous connections as a direct request from a client application to an object that lives on a remote network server. Because the connection is synchronous, the client application waits for the task to be complete, and is blocked until the server returns.

One of the limitations of the direct synchronous approach is the overhead involved in starting the worker and service objects. When a client is done with a worker, both the worker and service are destroyed.

In this profile there are five client processes with a 0 millisecond call delay on 100 calls per client, connected directly through one server machine to the Microsoft Jet database using ADO.

For this test run, the Overall Calls Per Second is 27.4 calls.


Customizing Application Performance Explorer

You can modify the APE components to more closely model your enterprise application. The complete Visual Basic source code and project files for each component (except Aemanagr) are located in the ..\Program Files\Microsoft Visual Studio\Common\Tools\Ape\Source subdirectory of your Visual Studio installation.

Note The compatible executables and DLLs for Application Performance Explorer are shipped in the ..\Program Files\Microsoft Visual Studio\Common\Tools\Ape\Source\AECompat directory. Except for the MTS Service (Aemtssvc.dll) and the Expediter (Aeexpdtr.dll), all components use Project, not Binary, compatibility.


Application Performance Explorer Common Tasks

This section contains step-by-step instructions for the most common Application Performance Explorer tasks. These tasks include the following.

Installing Application Performance Explorer

Resolving APE Problems

Monitoring MTS Transactions


Installing Application Performance Explorer

To use this release of Application Performance Explorer (APE), you should read each of the following topics to determine which are required for your APE installation.

  • Installing Microsoft® Transaction Server

  • Installing Microsoft® SQL Server™ with Service Pack 3

  • Configuring DCOM Access Permissions Using DCOMCNFG

  • Configuring RA Access Permissions Using RACMGR32

Installing Microsoft Transaction Server

You must install Microsoft Transaction Server (MTS) separately on all server machines. This includes standalone workstation configurations where all of APE runs on a single machine. APE MTS scenarios will not run without installing MTS.

After installing MTS and APE, you must install the APE MTS package on each server machine.

To install the APE MTS package on each server machine

  1. From the Start menu, point to Programs, Windows NT 4 Option Pack, Microsoft Transaction Server, and then click Transaction Server Explorer.

  2. In the left pane, expand the Microsoft Transaction Server folder, the Computers folder, and the MyComputer folder. Right-click the Packages Installed folder, point to New, and then click Package.

  3. In the Package Wizard dialog box, click Install pre-built packages.

  4. In the Select Package Files dialog box, click Add.

  5. In the Install from Package File dialog box, browse to Program Files\Microsoft Visual Studio\Common\Tools\Ape\Aemtssvc.pak, and then click Open.

  6. In the Select Package Files dialog box, click Next.

  7. In the Set Package Identity dialog box, click Next.

  8. In the Installation Options dialog box, click Finish.

Installing Microsoft SQL Server with Service Pack 3

If you want to test the performance of MTS scenarios that include Microsoft SQL Server, you must install Microsoft® SQL Server 6.5™ and Microsoft SQL Server Service Pack 3.

To install Microsoft SQL Server 6.5 with Service Pack 3

  1. Run the Microsoft® Visual Studio™ Installation Wizard.

  2. Click Server Applications and Tools (Add Only), and then click Next.

  3. Click Launch BackOffice Installation Wizard, and then click Install.

  4. In the BackOffice Business Solutions dialog box, click Custom, and then click Next.

  5. In the Disk Space Requirements dialog box, click Next.

  6. In the BackOffice Programs and Their Components dialog box, in the BackOffice Programs list, click SQL Server Developer Edition 6.5, clear check boxes for the other selections, and then click Next.

  7. In the Administrator and Service Accounts for BackOffice Programs dialog box, click Assign Accounts to create at least one SQL Server system administrator account name and password, and then click Next.

  8. In the Your BackOffice Installation dialog box, click Next and then follow the remaining instructions to install SQL Server 6.5.

Note This installation of SQL Server 6.5 includes Service Pack 3. You do not need to run a separate Service Pack 3 installation process.

Configuring Access Permissions Using DCOMCNFG

Note This topic applies to your APE installation only if you plan to use DCOM in your APE tests.

APE can use remote machines to offload service tasks during distributed performance tests. This situation occurs when you test with the following options.

  • The Use Remote Application Server check box is selected in the Application Service Connection Options dialog box.

  • A valid machine name is provided in the Server Name field.

  • APE server components have been set up on the named server.

  • A valid user has logged on to Microsoft® Windows NT® at the named machine.

Before APE can successfully use remote machines in distributed performance tests, you must use the DCOMCNFG utility to configure appropriate access permissions for each of the following APE machine configurations.

  • APE Service Manager machine (the main server machine that will host the Queue and Pool Managers).

  • APE Client Manager machine (the machine that the main APE Control Panel UI runs on).

  • APE Remote Client machine (client machines other than the APE Client Manager machine).

  • APE Remote Worker machine (worker machines other than the APE Service Manager machine).

Note These four APE machine configurations do not necessarily require four separate machines. You could use just two machines.

For each of the preceding machine configurations, you use DCOMCNFG to perform the following simple steps.

  1. Start the DCOMCNFG utility.

  2. Select the machine configuration (APE Service Manager, APE Client Manager, APE Remote Client, or APE Remote Worker) from within the DCOMCNFG application list.

  3. Click the Default Properties tab.

  4. Select the Enable Distributed COM on this computer check box.

  5. In the Default Authentication Level list, select None.

  6. In the Default Impersonation Level list, select Identity.

  7. Define access and launch permissions for each user of this service.

Note Each logged-on user of each service that will connect to the APE clients or services must have permission to access all APE components. To define these permissions, use the procedure "To define component access and launch permissions" at the end of this topic.

  8. Click the Applications tab.

  9. In the Applications list, select Ape Worker.

  10. Click Properties.

  11. In the Application Properties dialog box, click the Identity tab.

  12. Click The interactive user.

  13. Click OK.

  14. Select Ape Worker Provider and repeat Steps 10 through 13.

  15. Select Ape Queue Manager and repeat Steps 10 through 13.

  16. Select Ape Pool Manager and repeat Steps 10 through 13.

  17. Select Ape Logger and repeat Steps 10 through 13.

  18. Select Ape Server Manager and repeat Steps 10 through 13.

To define component access and launch permissions

  1. Click the Default Security tab.

  2. Under Default Access Permissions, click Edit Default.

  3. In the Registry Value Permissions dialog box, in the Name list, add each user name that will have access permissions for the APE Service Manager machine.

  4. In the Type of Access list, select Allow Access for each user permitted to access components.

  5. Under Default Launch Permissions, click Edit Default.

  6. In the Registry Value Permissions dialog box, in the Name list, add each user name that will have launch permissions for the APE Service Manager machine.

  7. In the Type of Access list, select Allow Launch for each user permitted to launch components.

Configure RA Access Permissions Using RACMGR32

After installing APE server components, you need to enable Remote Automation (RA) by completing the following.

  • Setting System Security Policy

  • Starting the Automation Manager

Setting System Security Policy

You must run the RA Connection Manager utility on each machine that will participate in your APE tests, and set the System Security Policy option to Allow All Remote Creates.

To set the System Security Policy option

  1. From the Start menu, click Run.

  2. Type Racmgr32.exe and click OK.

  3. Click the Client Access tab.

  4. Click Allow All Remote Creates.

  5. Click File and Exit.

Starting the Automation Manager

Before starting your APE test session, you must start the Automation Manager on each APE machine. Once Automation Manager is running, subsequent APE tests will run without intervention.

To start the Automation Manager

  • Browse to C:\Winnt\System32\Autmgr32.exe and double-click Autmgr32.exe.


Resolving APE Problems

Error Installing APETEST Database on SQL Server

Due to the variations in SQL Server configurations, it is possible for the installation of the APETEST database to fail. Before running the APE Database Setup Wizard again, you must remove any partially installed components.

For More Information For more information on using the APE Database Setup Wizard to install the APETEST database, see APE Database Setup Wizard.

To delete the physical device APETEST

  1. Start SQL Enterprise Manager.

  2. From the Manage menu, click Database Devices.

  3. Select APETEST.

  4. Click Delete Device.

To delete the APETEST database

  1. Start SQL Enterprise Manager.

  2. Open the following folders:

    <server name>\Database

  3. Select APETEST.

  4. Press the Delete key.

To delete the Apetest.dat file

  1. Using Windows Explorer, find and select the Apetest.dat file.

  2. Press the Delete key.

To check for the APE tables in the master database

  1. Start SQL Enterprise Manager.

  2. Open the following folders:

    <server name>\Databases\master\objects\tables

    The APE tables (ACCOUNT, ORDERDETAILS, and PRODUCTS) should not be in this list of master database tables. If they are, select and delete each one.

Full APETEST Database Log File

If your database is set up to log all I/O operations, the log file may rapidly fill up. This causes the database to stop accepting all new change requests, which in turn causes APE MTS transactions to abort.

To make space for new change logs, you must either back up or dump the log file. An easy way to dump the log file is to use Microsoft® SQL Server™ Enterprise Manager.

To use SQL Server Enterprise Manager to clear the APETEST log file

  1. Start Microsoft SQL Enterprise Server Manager.

  2. Select the APETEST database.

  3. Click Tools, and then click SQL Query Tool.

  4. In the Query list box, type the following line:
    dump transaction APETEST WITH NO_LOG
  5. Click Execute Query.


Monitoring MTS Transactions

To monitor MTS transactions

  1. In the Microsoft Management Console, in the tree view window, open the Microsoft Transaction Server folder.

  2. Open the Computers folder.

  3. Open the computer on which MTS is running, this will be My Computer if it is the local system.

  4. Click Transaction Statistics.

The statistic updates are refreshed automatically and sometimes take a few seconds.


Application Performance Explorer User Interface Reference

This section includes reference information on items in the Application Performance Explorer (APE) user interface. Click any of the links below for more information on the named item.

APE Database Setup Options Dialog Box

APE Database Setup Wizard

Application Performance Explorer Main Window

Application Service Connection Options Dialog Box

Callback (Client Options Dialog Box)

Concurrency (Client Options Dialog Box)

Configure Remote Client Machines Dialog Box

Configure Remote Worker Machines Dialog Box

CPU Task (Application Service Options Dialog Box)

Create Log File Dialog Box

Database Connection Options Dialog Box

Database Logon Dialog Box

Database Options Dialog Box

DB Task (Application Service Options Dialog Box)

Display (Administration Options Dialog Box)

General (Client Options Dialog Box)

Log Setup (Administration Options Dialog Box)

Logging (Administration Options Dialog Box)

Pooling (Application Service Options Dialog Box)

Return Data (Client Options Dialog Box)

Save Profile As Dialog Box

Send Data (Client Options Dialog Box)

Service Binding (Application Service Options Dialog Box)


APE Database Setup Options Dialog Box

Use this dialog box to select the Microsoft® SQL Server™ database.

Options

Connect to existing APE SQL Server database

Specifies use of an existing database. You will be prompted for the name of the SQL Server.

Install new APE SQL Server database

Specifies a new SQL Server database. The APE Database Setup Wizard will help you define the new server.


APE Database Setup Wizard

The APE Database Setup Wizard is the fastest way to create a new SQL Server database. The wizard consists of five main steps.

  1. Selecting a database server.

  2. Defining the system administrator login.

  3. Defining the destination directory.

  4. Defining the database owner.

  5. Finishing the database installation.

The wizard creates and installs the new APETEST database with one public user (the database owner) and the three APE test tables (Account, OrderDetails, and Products).

Note The APE Database Setup Wizard does not handle error conditions. Any database setup errors will be displayed using ODBC syntax.

The wizard includes a standard set of buttons along the bottom. These buttons control the following actions.

Button | Description
Help | Displays online Help for the Project Wizard.
< Back | Moves back one step in the process.
Next > | Moves forward one step in the process.
Finish | Creates all items and closes the wizard.
Cancel | Closes the wizard; no database is created.

Please Select a Server

This step specifies the kind of database server you want. There is only one option:

  • Microsoft SQL Server.

Server Administrator Login

This step defines the database administrator. The login identified here must belong to an authorized database administrator with full access permissions.

Login Id

Provides a place to specify the administrator's login name.

Password

Provides a place to specify the administrator's password.

Server

Provides a place to specify the name of the new database server.

Destination Directory

This step defines directory, device, and file size for the new database server.

Device ID

Provides a place to specify the unique device identification for the database and transaction logs.

Device Size (MB)

Defines the approximate size of the database.

Server Device Path

Provides a place to specify the directory where the database will be created.

Who's the DBO?

This step defines the database owner.

Confirm password

Confirms the database owner's password.

Log on dbo id

Provides a place to specify the database owner. The default is APETESTER.

Password for log on

Provides a place to specify the database owner's password.

Finish

This step completes the wizard and displays a confirmation message indicating that the database was successfully installed.


Application Performance Explorer Main Window

Use the APE Main Window to select the test Profile, start, stop, and close the test, and clear the log file. Additionally, you can click any of the five large buttons to display additional dialog boxes to completely configure your APE test.

Options

Clear

Clears the summary window. As a general guideline, you should delete the summary information prior to running another test. After deleting the summary information, the log file still retains all of the detailed run-time statistics.

Client Options

Displays the Client Options dialog box.

Client Service Connection

Displays the Application Service Connection Options dialog box.

Database Connection

Displays the Database Connection Options dialog box.

Database Options

Displays the Database Options dialog box.

Profile

Displays a list of available test profiles within the current profile collection. You can select any profile, modify it using the configuration dialog boxes, and then save it as a customized profile with a unique name.

To load a profile

  • Select a profile from the Profile list.

To save the selected profile

  • Click Save.

To delete the selected profile

  • Click Delete.

To select a different profile collection

  • From the File menu, click Open Profile Collection.

Service Options

Displays the Application Service Options dialog box.

Start

Begins the APE test based on the current profile settings.

Stop

Stops the currently running APE test. The run-time statistics up to the time of stopping remain available.


Application Service Connection Options Dialog Box

Use this dialog box to configure the remote application server.

Options

Connection Type

Specifies the type of connection used to connect to remote machines in order to launch the server components and remote clients. The available options include:

  • Distributed COM

  • Remote Automation

Note Every machine that hosts remote APE components (such as workers, services, or clients) must be running Microsoft® Windows NT® 4.0. The APE Manager machine must be running either Windows NT 4.0 or Windows® 95 (or higher).

Local (COM)

Specifies using APE server components on the local machine.

RA Protocol

Specifies the type of Remote Automation protocol for connecting to remote machines. The available options include:

  • TCP/IP

  • Named Pipes

Note To use Remote Automation, the Automation Manager must be running on every machine except the APE Manager machine.

Remote

Specifies using APE server components on a remote machine. When Remote is checked, you must specify a valid network server name.

Server Name

Provides a place to specify the name of the remote machine where the APE server components will run.


Callback (Client Options Dialog Box)

Use the Callback tab to configure the callback options.

Options

Register Callback Every Service Request

Specifies that clients will pass a callback object every time they make a service request.

Register Callback Only Once

Specifies that clients will pass a callback object only once.

Return Results Using Event Sinking

Specifies that results are returned to the clients by raising events rather than through a callback object.


Concurrency (Client Options Dialog Box)

Use the Concurrency tab to configure the number and location of client processes.

Options

Allow Externally Hosted Clients

Specifies whether additional client processes can be added to a test machine by other APE tests. If this check box is selected, other APE tests can start client processes on one or more of your test machines. If this check box is cleared, the test profile is essentially locked and the test will run with the configured client workload.

Client Processes per Machine

Specifies the number of client processes created on each machine.

Configure

Displays the Configure Remote Client Machines dialog box, where you can modify the list of remote client machines.

Use Remote Client Machines

Specifies the use of client processes on remote machines (instead of local).

When this check box is selected, you can configure Application Performance Explorer (APE) to start client processes on remote machines. Selecting this box displays the Configure Remote Client Machines dialog box, where you can create a list of remote machines.


Configure Remote Client Machines Dialog Box

Use this dialog box to specify each remote client machine that is available for the test.

Options

Add to Test

Adds the test machine entered in the Client Machine Name box to the list of client test machines available for the test.

Client Machine Name

Provides a place to specify the name of a remote client machine. Enter the name and then click Add to Test to add this client machine to the list of remote client test machines.

Client Test Machines

Displays the list of active remote client machines. To delete a listed machine, select the machine name and click Remove. To eliminate a listed machine from the test but still keep the machine in the list, clear the check box beside the machine name.

Remove

Removes the selected client test machine from the machines available for the test.

Use Client Manager Machine as Client Machine

Specifies that the client manager machine should also be used as a client machine (in addition to the listed client test machines). This option is the default when no remote clients are selected. However, when remote clients are specified, you can additionally use the client manager machine as a client.


Configure Remote Worker Machines Dialog Box

Use this dialog box to specify each remote worker machine name that is available for the test.

Options

Add to Test

Adds the test machine entered in the Worker Machine Name box to the list of available worker test machines.

Remove

Removes the selected worker machine from the available test machines.

Use Service Manager Machine as Worker Machine

Specifies that the job or pool manager machine should also be used as a worker machine. This option is the default when no remote workers are selected. However, when remote workers are specified, you can additionally use the service manager machine as a worker.

Worker Machine Name

Provides a place to specify the name of a worker machine. Enter the name and then click Add to Test to add this worker machine to the list of worker test machines.

Worker Test Machines

Displays the list of active worker machines. To delete a listed worker machine, select the machine name and click Remove. To eliminate a listed machine from the test but still keep the machine in the list, clear the check box beside the machine name.


CPU Task (Application Service Options Dialog Box)

Use the CPU Task tab to specify the duration and characteristics of a CPU resource workload on machines hosting workers.

Options

Simulate CPU Based Task

Specifies a CPU task as part of the test.

Sleep Period (ms)

Specifies the sleep period in milliseconds for the CPU task.

+/-

Specifies the ms variation — using Sleep Period (ms) as a base — for the CPU task’s sleep period.

Task Duration (ms)

Specifies the duration in ms for each CPU task. Due to the possibility of multi-tasking other non-APE application tasks, this value is not the same as specifying that each CPU task should be fully active for the defined duration. Instead, each task will start and stop according to the specified duration, using the CPU resources as available during that time.

+/-

Specifies the ms variation — using Task Duration (ms) as a base — for the CPU task.


Create Log File Dialog Box

Use this dialog box to specify the name and location of the log file.

Options

File name

Provides a place to specify the log file name.

Save

Creates the new log file in the specified folder.

Save as type

Lists the available log file types (*.log is recommended).

Save in

Lists the folders where you can save the log file.

Save in list

Lists the items in the current Save folder.


Database Connection Options Dialog Box

Use this dialog box to specify the database access technology.

Options

Db Access Options

Specifies the available database access methods. The listed options include:

  • ADO

  • RDO

  • DAO

  • ODBC API


Database Logon Dialog Box

Use this dialog box to specify the user ID and password for connecting to a database.

Options

Password

Provides a place to specify the password. The default password for the APETEST database is all blanks.

User ID

Provides a place to specify the user ID. The default user id for the APETEST database is APETESTER.


Database Options Dialog Box

Use this dialog box to select the database engine.

Options

Microsoft Jet

Specifies that the test should use the Microsoft® Jet database (Apetest.mdb).

Microsoft SQL Server

Specifies that the test should use the Microsoft® SQL Server™ database.


DB Task (Application Service Options Dialog Box)

Use the DB Task tab to configure the database resource workload.

Options

Database Query

Specifies that a named database query will be part of the test. When Database Query is checked, you must specify a valid query name.

Perform Database Task

Specifies that database access is part of the test.

Query

Provides a place to specify the name of the query that will be part of the test.

SQL

Displays the SQL statements that compose the query.

Write Distributed MTS Transactions

Specifies that the test uses Microsoft® Transaction Server transactions.


Display (Administration Options Dialog Box)

Use the Display tab to request views of various APE progress statistics as the test is running. Displaying progress requires considerable machine resources and may distort the run times of your test.

Options

Display Client Progress

Specifies that you want to display the client progress.

Display Expediter Progress

Specifies that you want to display the expediter progress.

Display Job/Pool Manager Progress

Specifies that you want to display the queuing and pooling progress.

Display MTS Progress

Specifies that you want to display the MTS progress.


General (Client Options Dialog Box)

Use the General tab to configure the delay between calls and how long the test will run.

Options

Continue Until Stopped

Specifies that the test will run continuously until you click Stop.

Delay After Calls: (ms)

Specifies the time in ms that each client process waits between calls.

+/-

Specifies the amount of delay variation (using Delay After Calls as a base) for data returned to the client objects.

Number of Calls

Specifies the duration of the test as a certain number of calls.

Number of Minutes

Specifies the duration of the test in minutes.


Log Setup (Administration Options Dialog Box)

Use the Log Setup tab to configure the logging options.

Options

Browse

Displays the Create Log file dialog box, where you can select an existing log file.

Clear Log

Deletes the current log file.

Flush log file to disk every: (KB)

Specifies that the log records are held in memory up to the kilobyte threshold, at which time the log records are written to the disk log file. Writing log records directly to disk can slow the currently running APE test. Holding the log records in memory postpones the disk I/O and helps your test run faster.

Log File

Specifies the log file name. All log records and error messages generated by the components are written to this file.

Maximum log file size: (MB)

Specifies the maximum disk space that will be used for the log file. If the log file size is exceeded, the earliest log entries are deleted and the newest log entries are appended to the end of the file.

Overwrite the file at the start of each new APE session

Indicates that the APE Manager should overwrite the log file every time a test runs. If this check box is cleared, the APE Manager appends all new log records to the existing file.

View

Displays the current log file in Notepad.


Logging (Administration Options Dialog Box)

Use the Logging tab to select the event types that are logged.

Options

Log Client Events

Specifies that client events are logged.

Log Database Events

Specifies that database events are logged.

Log Expediter Events

Specifies that expediter events are logged.

Log Job/Pool Manager Events

Specifies that queuing and pooling events are logged.

Log MTS Events

Specifies that MTS events are logged.

Log Summary Information

Specifies that the APE Manager prints summary information to the log file. The summary information includes a profile summary that is at the beginning of the log, and a performance summary that is at the end of the log.

Log Worker Events

Specifies that worker events are logged.


Pooling (Application Service Options Dialog Box)

Use the Pooling tab to configure the number and characteristics of the pooled resources.

Options

Pooled Objects per Machine

Provides a place to specify how many pooled objects per machine you want to test.

Pooled Workers per Machine

Provides a place to specify how many pooled workers per machine you want to test.

Use Job Manager

Specifies to use both job queuing and worker pooling.

Use Pool Manager

Specifies to use object pooling only (and not job queuing).

Use Pre-Created, Pooled Resources

Specifies that the worker resources should be pooled and recycled as needed. Without this option, the client needs to instantiate a worker each time it's needed. Your test (and your application) will run more efficiently using pooled resources.

Use Remote Worker Machines

Specifies the use of workers on remote machines. A Configure button displays the Configure the Worker Machines dialog for defining the worker machine names.


Return Data (Client Options Dialog Box)

Use the Return Data tab to specify how data is returned from the service objects.

Options

Pass Results to Client (via callback object)

Specifies that data is to be returned to the client on every client call.

Variant Array

Returns a Variant Array data type to the client. Using arrays instead of collections is recommended.

Variant Collection

Returns a Variant Collection data type to the client. This is not a recommended option. It is only provided to demonstrate the cost of passing collections.

Bytes/Row

Specifies how many bytes per row are returned to the client objects.

+/-

Specifies the amount of data size variation (using Bytes/Row as a base) for data returned to the client objects.

Rows

Specifies how many rows are returned to the client objects.

+/-

Specifies the number of rows variation (using Rows as a base) for data returned to the client objects.


Save Profile As Dialog Box

Use this dialog box to enter a custom name for the modified profile.

Options

Profile Name

Provides a place to specify the new profile name.


Send Data (Client Options Dialog Box)

Use the Send Data tab to specify how data is sent to the service objects.

Options

Pass Data to Services

Specifies that client processes pass data to the service objects with each call. In the asynchronous model, the data is passed through the job manager and a worker with each service request. In the synchronous model, data is passed only through a worker.

Variant Array

Passes a Variant Array data type to the service objects. Using arrays instead of collections is recommended.

Variant Collection

Passes a Variant Collection data type. This is not a recommended option. It is only provided to demonstrate the cost of passing collections.

Bytes/Row

Specifies how many bytes per row are passed to the service objects.

+/-

Specifies the amount of data size variation (using Bytes/Row as a base) for data passed to the service objects.

Rows

Specifies how many rows are passed to the service objects.

+/-

Specifies the number of rows variation (using Rows as a base) for data passed to the service objects.


Service Binding (Application Service Options Dialog Box)

Use the Service Binding tab to configure early or late binding for service objects.

Options

Early Bind Services

When selected, this box specifies that services will be early bound. You should test the performance difference between early binding and late binding with your application. Early binding will definitely improve the speed of your application.

Preload Services

When selected, this box specifies that workers load the service object before timing the test. By choosing this option, the time needed to instantiate the service object is not included in the test duration. Preload Services has no effect if the Retain Service References box is not selected.

Retain Service References

When selected, this box specifies that workers keep a reference to any service object as they are needed. The worker will only have to instantiate a service object the first time it’s used. After that first time, the worker uses its stored reference to the service to accomplish tasks.


Application Performance Explorer Error Messages

This section contains reference information on Application Performance Explorer error messages. These error messages include the following.

A client is being removed from a test in progress

A CPU task or a Database task must be selected

Access denied to <NAME>

Additionally, <NUMBER> externally hosted client(s) joined the test

All requested clients were successfully created

All requested workers were successfully created

An error occurred changing server connection settings

An error occurred instantiating AEExpediter.Expediter

An error occurred reading from the registry

An error occurred writing to the registry

Because critical errors occurred, Application Performance Explorer will close

Call retries exhausted

Could not connect to server using Remote Automation

Could not create account object

Could not create MTS Service

Could not create object

Could not create or configure a client on machine

Could not create or configure a worker on machine

Could not create or configure local client object

Could not create or configure local worker object

Could not create workers

Disk full, logging turned off

Error allocating handle

Error clearing the log file

Error closing cursor

Error connecting driver

Error deleting profile

Error disconnecting driver

Error ending transaction

Error executing query

Error fetching record

Error freeing handle

Error in Execute method

Error opening service connection

Error parsing database query

Error performing database query

Error posting to account

Error setting attribute

Failed to create AEServerMgr.ServerMgr on the remote machine

Invalid Client Machine Name

Invalid profile name

Invalid remote client machine

Invalid remote worker machine

Invalid Server Name

Invalid Worker Machine Name

Make sure correct version of AESvrMgr.exe is installed and registered

Make sure correct version of RacReg32.dll is installed and registered

No clients could be created

No protocol was specified

No server machine name was specified

No Workers could be created

OLE collision retries exhausted

Only <NUMBER> client(s) were created and configured

Only <NUMBER> workers were created and configured

Please choose a valid log file

Please start the APE Manager that has control of the server first

Please wait for other APE Manager sessions to finish tests first

PoolMgr rejection retries exhausted

Test Started

The path specified for the log file is invalid

Unexpected error

Will attempt to connect to local server


A client is being removed from a test in progress

Client log records are retrieved at the end of a test run. By removing a client during the test, you are removing all statistics for this client from the log results.


A CPU task or a Database task must be selected

An APE profile must include either a CPU or database task.

Try the following:

  • Include a CPU-based task by clicking the CPU Task tab on the Application Service Options dialog box. Select the Simulate CPU Based Task check box.

  • Include a database-based task by clicking the DB Task tab on the Application Service Options dialog box. Select the Perform Database Task check box.


Access denied to <NAME>

The remote computer <name> did not allow APE to run.

Try the following:

  • Confirm that the remote computer is still turned on and logged on to the network.

  • Confirm that a valid APE (client or server) installation exists on the computer.

  • If the remote computer is logged on correctly, confirm that AutMgr32.exe is running to support Remote Automation calls and that the necessary security rights have been set to allow for the use of distributed COM.

For More Information For information on installing APE and configuring security rights, see Installing Application Performance Explorer.


Additionally, <NUMBER> externally hosted client(s) joined the test

Indicates the number of clients that have been added to the test run.


All requested clients were successfully created

The initialization of an APE profile, which involves the creation of client processes (either local or remote), succeeded.


All requested workers were successfully created

The initialization of an APE profile, which involved the creation of worker processes (either local or remote) succeeded.


An error occurred changing server connection settings

An error occurred changing the remote connection settings for some APE components.

Try the following:

  • Confirm that a valid APE (client or server) installation exists on the computer.

  • Confirm that the necessary DCOM and Remote Automation files and settings exist on the computer.

  • If using Remote Automation, make sure the Automation Manager is running on the named computer.


An error occurred instantiating AEExpediter.Expediter

APE requires the object, AEExpediter, to be registered correctly and allow the usage of callbacks.

Try confirming that all local APE components and any remote installations being used in the current profile were installed successfully. If not, you may need to reinstall the APE client or server, as appropriate.

For More Information For information on installing APE, see Installing Application Performance Explorer.


An error occurred reading from the registry

APE expects to persist some settings in the Windows Registry. In order to do this, the current user must have security rights to access the Registry.

Try confirming that your security rights on the local computer allow reading and writing to the Windows Registry.


An error occurred writing to the registry

APE expects to persist some settings in the Windows Registry. In order to do this, the current user must have security rights to access the Registry.

Try confirming that your security rights on the local computer allow reading and writing to the Windows Registry.


Because critical errors occurred, Application Performance Explorer will close

A critical error occurred in APE.

Try confirming that all local APE components and any remote installations being used in the current profile were installed successfully. If not, you may need to reinstall the APE client or server, as appropriate.

For More Information For information on installing APE, see Installing Application Performance Explorer.


Call retries exhausted

The APE Manager has reached its maximum number of attempts to call a component to stop the currently running test. This usually occurs when the component is too busy running the test to respond to the APE Manager request to stop.


Could not connect to server using Remote Automation

To use Remote Automation (unlike distributed COM), you must choose which protocol to use.

Try the following:

  • From the APE Manager View menu, click Service Connection Options, then select a valid protocol from the RA Protocol list.


Could not create account object

An error occurred while trying to create the MTS Account Service.

Try the following:

  • Confirm that MTS is installed correctly on the computer running APE Services. This is determined using the Application Services Options dialog box within the APE Manager.

  • Confirm that the APE Package has been installed correctly into MTS.

  • Confirm that you have security rights to call the MTS package.

For More Information For information on installing APE, see Installing Application Performance Explorer.


Could not create MTS Service

An error occurred while trying to start the MTS Service.

Try the following:

  • Confirm that MTS is installed correctly on the computer set up to run the APE Services. APE Services are configured using the Application Services Options dialog box within the APE Manager.

  • Confirm that the APE Package has been installed correctly into MTS. See the APE setup help for information on how to do this.

For More Information For information on installing APE, see Installing Application Performance Explorer.


Could not create object

This standard error is reported when there has been a failure in creating a COM object — in this case while the APE Manager is preparing to start a test.

Try confirming that the installation of all local APE components and any server installations being used in the current profile were successful. If not, you may need to reinstall either the APE client or server, as appropriate.

For More Information For information on installing APE, see Installing Application Performance Explorer.


Could not create or configure a client on machine

An error occurred when trying to communicate with a client process on the named computer.

Try the following:

  • Confirm that a valid APE (client or server) installation exists on the computer.

  • Confirm that the necessary DCOM and Remote Automation files and settings exist on the computer.

  • If you are using Remote Automation, make sure the Automation Manager is running on the named computer.

For More Information For information on configuring APE components, see Installing Application Performance Explorer.


Could not create or configure a worker on machine

An error occurred when trying to create a worker on the named computer.

Try the following:

  • Confirm that the remote computer is still turned on and logged on to the network.

  • Confirm that a valid APE (client or server) installation exists on the computer.

  • Confirm that you have valid security rights to the remote computer.

  • Confirm that the necessary DCOM and Remote Automation files and settings exist on the computer.

For More Information For information on installing APE components, see Installing Application Performance Explorer.


Could not create or configure local client object

An error occurred when trying to create a client process on the local computer. The local computer is the computer running the APE Manager.

Try confirming that the local APE installation was successful.

For More Information For information on installing APE components, see Installing Application Performance Explorer.


Could not create or configure local worker object

An error occurred when trying to create a worker on the local computer. The local computer is the computer running the Pool Manager or Job Manager.

Try ensuring that a valid APE (client or server) installation exists on the computer running the Pool Manager or Job Manager.


Could not create workers

An error occurred while creating a worker on the local computer.

Try the following:

  • Confirm that a valid APE (client or server) installation exists on all computers on which workers are specified. Confirm that the necessary DCOM and Remote Automation files and settings exist on those computers.

  • If using Remote Automation, make sure the Automation Manager is running on those computers.


Disk full, logging turned off

The disk with the logging file has run out of space.

Try the following:

  • Change the location of the logging file to a disk with adequate space.

  • Free up space on the current disk.


Error allocating handle

This is an ODBC API error that occurred while allocating a handle.

Try checking the ODBC API documentation for more information.


Error clearing the log file

An error occurred while accessing the APE log file.

Try the following:

  • Ensure any other applications or instances of APE are not accessing the log file.

  • Confirm that you have valid security rights to the log file.


Error closing cursor

This is an ODBC API error that occurred while closing the database cursor while running the selected APE profile.

Try checking the ODBC API documentation for more information.


Error connecting driver

This is an ODBC API error that occurred while connecting to the database driver specified by the current APE profile.

Try the following:

  • Confirm that the database you are accessing is valid and installed without error. If it is the APE Access database (Apetest.mdb), you can replace it with the original file from your original product CDs.

  • See the ODBC API documentation for more information.


Error deleting profile

An error occurred while deleting a profile from the APE profile Aemanagr.ini file.

Try the following:

  • Confirm that any other applications or instances of APE are not accessing the Aemanagr.ini file.

  • Confirm that you have valid security rights to the Aemanagr.ini file.


Error disconnecting driver

This is an ODBC API error that occurred while disconnecting from the database driver specified by the current APE profile.

Try the following:

  • Confirm that the database that you are accessing is valid and installed without error. If it is the APE Access database (Apetest.mdb), you can replace it with the original file from your original product CDs.

  • See the ODBC API documentation for more information.


Error ending transaction

This is an ODBC API error that occurred while ending a database transaction.

Try the following:

  • Confirm that the database that you are accessing is valid and installed without error. If it is the APE Access database (Apetest.mdb), you can replace it with the original file from your original product CDs.

  • See the ODBC API documentation for more information.


Error executing query

This is an ODBC API error that occurred executing the database query specified by the current APE profile.

Try the following:

  • Confirm that the database that you are accessing is valid and installed without error. If it is the APE Access database (Apetest.mdb), you can replace it with the original file from your original product CDs.

  • See the ODBC API documentation for more information.


Error fetching record

This is an ODBC API error that occurred fetching a record from the APE database specified by the current APE profile.

Try the following:

  • Confirm that the database that you are accessing is valid and installed without error. If it is the APE Access database (Apetest.mdb), you can replace it with the original file from your original product CDs.

  • See the ODBC API documentation for more information.


Error freeing handle

This is an ODBC API error that occurred freeing the database handle while running the selected APE profile.

Try checking the ODBC API documentation for more information.


Error in Execute method

An error occurred in the Execute method of a Service component. The Execute method is called by the worker component to perform the tasks specified in the profile. Error text appended to this error message describes the specific error that occurred.


Error opening service connection

An error occurred while trying to establish a database connection. Error text appended to this error message describes the specific error that occurred.

Try the following:

  • If you are using the Jet database, make sure that the Apetest.mdb database exists at the location where APE is installed on all the computers that provide workers.

  • If you are using a SQL Server database, make sure that the SQL Server database is set up correctly and that the correct user ID and password is specified at the start of the test.

For More Information For information on installing the APE SQL Server database, see Installing Application Performance Explorer.


Error parsing database query

An error occurred while replacing the random elements of the selected SQL query. The random elements are text of the form "%m-n%" which are replaced by a random integer value in the range m through n.

Try confirming that the random elements have been specified using the correct syntax.


Error performing database query

An error occurred while executing a SQL query on the selected database. Error text appended to this message describes the specific error that occurred.

Try the following:

  • Confirm you have sufficient access rights to perform the specified query.

  • Read the documentation for the selected data access method for more information.


Error posting to account

An error occurred while trying to create the MTS Account Service.

Try the following:

  • Confirm that MTS is installed correctly on the computer set up to run the APE Services. This is configured using the Application Services Options dialog box within the APE Manager.

  • Confirm that the APE Package has been installed correctly into MTS. See the APE setup help for information on how to do this.

For More Information For information on installing APE components, see Installing Application Performance Explorer.


Error setting attribute

This is an ODBC API error that occurred while setting an ODBC attribute.

Try checking the ODBC API documentation for more information.


Failed to create AEServerMgr.ServerMgr on the remote machine

APE requires the file, AESvrMgr.exe, to support the calling of server processes.

Try the following:

  • If the error text directs you to the lack of response or failure to connect to a specified computer, check that the computer is turned on and logged onto the network.

  • If the remote computer is logged on correctly confirm that AutMgr32.exe is running on it to support Remote Automation calls and that the necessary security rights have been set to allow for the use of distributed COM.

  • Confirm that the installation of all local APE components and any remote installations being used in the current profile were successful. If not, you may need to reinstall either the APE client or server, as appropriate.

For More Information For information on installing APE components, see Installing Application Performance Explorer.


Invalid Client Machine Name

A client machine name specified in the Configure Remote Client Machines dialog box cannot be found.

Try confirming that a valid network computer is named, or enter the name of a valid network computer (with either an APE client or server installed) in the Client Machine Name box.


Invalid profile name

Profile names can be up to 60 characters long and must not contain the "]" character.

Try retyping the profile name.


Invalid remote client machine

A remote client machine name specified in the Configure Remote Client Machines dialog box cannot be found.

Try confirming that the machine name is a valid network computer, or enter another name of a valid network computer (with an APE client installed).


Invalid remote worker machine

A remote worker machine name specified in the Configure Remote Worker Machines dialog box cannot be found.

Try confirming that the machine name is a valid network computer, or enter another name of a valid network computer (with an APE server installed).


Invalid Server Name

The server name specified in the Application Service Connection Options dialog box cannot be found.

Try the following:

  • From the APE Manager View menu, click Service Connection Options. In the Server Name box, enter the name of a valid network computer with either an APE client or server installed.

  • Confirm that the remote computer is running AutMgr32.exe if you are using Remote Automation and have the necessary security rights set to allow for the use of distributed COM.


Invalid Worker Machine Name

A worker name specified in the Configure Remote Client Machines dialog box cannot be found.

Try entering the name of a valid network computer (with either an APE client or server installed) in the box titled Worker Machine Name.


Make sure correct version of AESvrMgr.exe is installed and registered

APE requires the file, AESvrMgr.exe, to support the calling of server processes.

Try confirming that all local APE components were installed successfully. If not, you may need to reinstall an APE client or server, as appropriate.

For More Information For information on installing APE components, see Installing Application Performance Explorer.


Make sure correct version of RacReg32.dll is installed and registered

APE requires the file, RacReg32.dll, to support Remote Automation and distributed COM.

Try confirming that all local APE components were installed successfully. If not, you may need to reinstall an APE client or server, as appropriate.

For More Information For information on installing APE components, see Installing Application Performance Explorer.


No clients could be created

APE failed to create the number of clients specified by the current profile.

Try the following:

  • Close down other applications running on the computer to free up memory or resources.

  • Confirm that a valid APE (client or server) installation exists on the computer.

For More Information For information on installing APE components, see Installing Application Performance Explorer.


No protocol was specified

To use Remote Automation, rather than distributed COM, you must choose which protocol to use.

Try the following:

  • From the APE Manager View menu, click Service Connection Options. Select a valid protocol from the RA Protocol list.


No server machine name was specified

You did not specify a valid server name in the Application Service Connection Options dialog box.

Try the following:

  • From the APE Manager View menu, click Service Connection Options. In the Server Name box, enter the name of a valid network computer with either an APE client or server installed.

  • Confirm that this remote computer is running AutMgr32.exe if you are using Remote Automation and have the necessary security rights set to allow for the use of distributed COM.


No Workers could be created

An error occurred when trying to create APE worker processes.

Try confirming that a valid APE (client or server) installation exists on the local computer.

For More Information For information on installing APE components, see Installing Application Performance Explorer.


OLE collision retries exhausted

A client has reached the maximum number of attempts to communicate with another component during the running of the current APE profile.

Try the following:

  • Lower the number of clients to see if it is a limitation of the computer or the speed of network.

  • Increase the number of workers.

  • Close down other applications running on the computer. This may help identify memory or other hardware resource constraints.


Only <NUMBER> client(s) were created and configured

Some of the remote computers set up to run APE client processes were unavailable during an APE profile initialization.

Try the following:

  • Confirm that the remote computers are all still turned on and logged on to the network.

  • Confirm that a valid APE (client or server) installation exists on the computer.

  • Confirm that you have valid security rights to the remote computer.

  • Confirm that the necessary DCOM and Remote Automation files and settings exist on the computer.


Only <NUMBER> workers were created and configured

Some of the remote computers set up to run APE worker processes were unavailable during an APE profile initialization.

Try the following:

  • Confirm that the remote computers are all still turned on and logged on to the network.

  • Confirm that a valid APE (client or server) installation exists on the computer.

  • Confirm that you have valid security rights to the remote computer.

  • Confirm that the necessary DCOM and Remote Automation files and settings exist on the computer.


Please choose a valid log file

The currently specified log file is invalid.

Try confirming that the path for the log file is valid. To check this, from the APE Manager View menu, click Admin Options, and then click the Log Setup tab and verify the path name.


Please start the APE Manager that has control of the server first

This message will normally appear when you have another instance of the APE Manager already running.

Try the following:

  • Close down the version of APE that you are currently running and use the instance that was already running.

  • If the preceding does not fix this problem, or no other instance seems to be already running, start up the Windows Task Manager and confirm that another instance isn't still in memory. If another instance is in memory, close it down and try again.


Please wait for other APE Manager sessions to finish tests first

You may have another instance of the APE Manager already running a profile.

Try waiting for the other instance of APE that is currently running to finish its session, or stop it using the APE Manager Stop button.


PoolMgr rejection retries exhausted

A client component has reached its maximum number of retries to get a worker from the Pool Manager. Whenever the Pool Manager has no available workers, it rejects the client's request for a worker. The client then waits for some time and retries the request. This cycle continues until the Pool Manager has a worker to assign to the client or the client reaches its maximum number of retries, whichever comes first.

Try increasing the number of workers.


Test Started

An APE test run using the selected profile has begun.


The path specified for the log file is invalid

The currently specified log file path is invalid.

Try the following:

  • From the APE Manager View menu, click Admin Options. Click the Log Setup tab and enter a valid path and filename for the APE log file into the Log File text box.


Unexpected error

An unexpected error occurred in APE.

Try the following:

  • Close and restart the APE Manager.

  • If the problem persists, confirm that the installation of all local APE components and any remote installations being used in the current profile were successful. If not, you may need to reinstall an APE client or server, as appropriate.

For More Information For information on installing APE components, see Installing Application Performance Explorer.


Will attempt to connect to local server

An attempt to connect to a remote server was unsuccessful. APE will connect to a local server instead.


Topic Not Found

No matching topic was found. Some possible reasons and remedies are as follows:

  • The file (.chm) that contains the destination topic is not in the expected location.

    Remedy: Rerun the MSDN Library Setup.

  • The file (.chm) that contains the current topic or destination topic is out-of-date and the link is no longer valid.

    Remedy: Install the latest version of the MSDN Library.

  • The MSDN Library CD is not in the CD-ROM drive.

    Remedy: Insert the MSDN Library CD into the CD-ROM drive.