Cincom® ObjectStudio® Benchmark Framework

The ObjectStudio Benchmark Framework (OBF) is a benchmark and observation tool that developers can use to monitor the performance of their application and also observe data to see what is actually happening within their applications.

The OBF is transactional based. This means that you define the starting of the benchmarking/observation with a begin: message and terminate it with an end message.  Since you can nest the begin: and end messages, you can create multiple nested observation transactions. During each transaction, you can collect data that you add to your logging through an addData: message.

Classes

The OBF consists of only two classes:

• BenchmarkExplorerController: a small GUI to activate, deactivate and view the benchmarking
• Benchmark: the class that collects all data

BenchmarkExplorerController

When the OBF is loaded, you get a program icon for the Benchmark Explorer on the ObjectStudio Main Window:

When you open the Benchmark Explorer, you get the following window:

On the left, we have a tree view that will show all observed transactions. Each transaction will be a node in the tree view.

On the right, are a range of buttons. The first one is:

• Activate/Deactivate: This starts/stops the benchmark/observation.  Activating and deactivating the Benchmarking can also be done programmatically.
• Reset: This will clear the tree view.
• Refresh: Pushing this button will refresh the tree view.
• Automatic: When this checkbox is active, the refresh button is disabled and the tree view is updated each time a main transaction is ended.
• The Total Time field will show you how long the benchmarking is running or ran.
• Close: This will close the Benchmark Explorer.

Using the Benchmark Class

To activate the benchmarking, you add:

Benchmark activate

or

Benchmark isActive: true.

To deactivate the benchmarking, you add:

Benchmark deactivate

or

Benchmark isActive: false

Start Benchmark

To start a benchmark transaction, you add one of the following instructions:

Benchmark begin: ‘some string’.

or

Benchmark begin: ‘some string’ with:  anObject.

or

Benchmark begin: ‘some string’ with:  anObject with: anotherObject.

or

Benchmark begin: ‘string’ with: obj1 with: obj2 with: obj3.

or

Benchmark begin: ‘string’ with: obj1 with: obj2 with: obj3 with: obj4.

With one of these instructions, the benchmarking transaction is started. The first argument is used as the label for the transaction and the other with: arguments are added as observed information for the transaction.

Add Data to Benchmark

Adding data to the benchmark transaction is done through the following code:

Benchmark addArrayOfData: anArrayOfObjects.

or

Benchmark addData: anObject.

With the first example, a collection of objects are added to the observation. With the last one, only one object is added to the observation.

End Benchmark

To end a benchmark transaction, we use:

Benchmark end.

or

Benchmark endWith: anObject.

or

Benchmark endWith: obj1 with: obj2.

or

Benchmark endWith: obj1 with: obj2 with: obj3.

The first example simply ends the transaction. The other ones end the transaction but first add the additional arguments to the observations.

Reset Benchmark

We have a benchmark reset method that will remove all observations from the benchmark.

Example and Use of the Benchmark Explorer

We have created a small example that will read the main page of three sites (CNN, Internet Movie Database and BBC). From the main page, we’ll observe the url and the page source.

From the main page source, we’ll search all href occurrences and read those pages. From these referenced pages, we’ll observe the url, the page source and the href references of these pages.

So we have a run method that contains:

Benchmark begin: 'Start Execution'.
self fetchSites:  siteCollection readRefs: true.
Benchmark endWith: ‘End of Execution’.

fetchSites: siteCollection readRefs: aBoolean
       | siteContents refs |
       siteCollection do: [:anUrl |
                Benchmark begin: 'Read Site ' , anUrl with: anUrl.
                siteContents := self getUrl: anUrl.
                Benchmark addData: siteContents.
                refs := self searchHrefsIn: siteContents.
                aBoolean
                        ifTrue: [self fetchSites: refs readRefs: false]
                        ifFalse: [Benchmark addData: refs].
                Benchmark end]

At the start, we begin our benchmark transaction, and we go into the fetchSites:readRefs: method. The first argument is #(‘http://www.cnn.com’ ‘http://www.imdb.com’ ‘http://www.bbc.co.uk’), and the second argument is true since we want to get the details of the href references.

In the fetchSites:readRefs: method, we see that before we read the first site, we start a new benchmark transaction and feed it the url as an observation. After we get the contents of the url, we add that to the observation data. We then search for all href references, and if we want to get the source of these urls, we start an additional transaction. If we don’t want the source, we simply add the collection of references to the observation. Then we end the subtransaction without further observations.

After all three websites are processed, we end the main transaction and add an additional string containing ‘End of Execution’ to the main transactions observation.

In the Benchmark Explorer, we start with an empty tree view. After the main transaction is done, we have the following screen:

After each transaction you’ll see a time. This is the time that was required to execute the complete transaction.  If we open some tree nodes, you’ll get something that looks like this:

Double click on one of the tree nodes, and an inspector will pop up. The inspector will be an array from the data we observed during the benchmarking transaction.  In our current example, this will be an array with three elements:

1. Contains the url we examined: http://rss.imdb.com/daily/died/.
2. Contains the contents of this url. From the screenshot, you’ll see that in 1900, Oscar Wilde died on this date (November 30) in Paris.
3. Contains an ordered collection with the href references. In our case, this is just one element ‘/xsl/rss’.

Remarks

You can use the begin, addDate and end message from the Benchmark class. The class will check whether it is activated or not, so you don’t need to test for this in your own code. Since the Framework collects data, the memory consumption may be considerable. Keep this in mind.

If you want to use your code without having the OBF as a prerequisite, then use the following protocol:

#{Benchmark} ifDefinedDo: [ :aClass | aClass begin].