The Controller

The controller is a flexible and powerful way for users to define a slice of execution which they want to profile by Intel® SDE or an Intel® SDE based tool. The controller component delivers controller events, and the tool is responsible to support the events, and define its behavior upon getting the event.

The controller supports a sophisticated definition for alarms and events. The user defines the controller events in a control chain. This defines a set of alarms which will trigger the events to be fired by the controller. All Intel® SDE active tools will receive these events, and act accordingly.

By default, if no start event is defined by the user in the command line, then a default start event is armed for each thread.

The controller provides a set of predefined alarms which include:

Name

Description

icount

count instructions where all the repeat string iterations are counted

instcount

count instructions where each repeat string instruction is count as a single count

address

a real virtual address or a function name (symbol) or image+offset

ssc

execution of a code sequence consisting of 2 instructions where the first has an immediate that identifies this marker

itext

execution of an instruction denoted by a sequence of raw bytes which is interpreted as an instruction

int3

execution of an embedded INT3 instruction

isa_extension

execution of an instruction which belong to this XED ISA extension name

isa_category

execution of an instruction which belong to this XED ISA category name

isa_category

execution of an instruction which has this XED ICLASS name

enter_func

entering to a function with this name

exit_func

returning from a function with this name

image_load

the time the image load callback was detected by pin

image_enter

execution of the first instruction in the image

cpuid

execution of a CPUID instruction with a special input (as defined by the input registers)

magic

execution of a code sequence also used by other simulation tools which has two input values that identify the marker

pcontrol

entering the MPI_Pcontrol function with specific string argument that identify the marker

timeout

number of seconds to trigger event

system_call

execution of a system call with an input value as the system call number

interactive

the user interactively send the event to the process from another window/shell on the same machine

Details

The controller exposes a knob: -control which gets the actual definition in a string argument. This knob can be defined multiple time in the Intel® SDE command line. This is a flexible way for the user to control the region being profiled by the tool.

Each instance of the “-control” knob, defines an alarm-chain as:

% sde -control <event>:<alarm>:<value>[:count<int>][:tid<int>][:bcast] ...

A full definition of the syntax, and the available alarm-types can be found below.

This defines that the <event> will be fired by the controller, once the <alarm> identified by the <value> is executed. If :count<int> is used, the event will be fired only when the alarm identified by the <value> is executed for the <int> time.

Examples:

% sde -skx -mix -control start:icount:100 -- myapp

The controller will fire a start event once 100 instructions are executed, and this will trigger the start of the collection of mix histogram analysis.

% sde -skx -mix -control stop:address:foo:count3 -- myapp

The collection of mix histogram analysis will start at the start of the process, and the controller will fire a stop event, once we reach the symbol ‘foo’ for the 3rd time.

Multiple alarms in a chain

Each instance of the -control knob defines, an alarm chain (see description below).

Alarm chain is a sequence of alarms, separated by the ‘,’ character, where each alarm is activated only after the previous one was fired. So the ‘,’ character applies an order between the proceeding alarm and the following alarm .

Example:

% sde -skx -mix -control start:icount:1000,stop:icount:2000 -- myapp

In this example the start event will be fired after 1000 instructions and the stop event after 2000 more instructions (i.e. at 3000).

The controller knob can be specified multiple times, each defining an independent alarm-chain.

% sde -skx -mix -control start:icount:1000 -control stop:icount:2000 -- myapp

In this example the start event will be fired after 1000 instructions and the stop event after 2000 instructions from the start of the run.

In addition, one can set the entire chain to start only after some other chain has finished, using the “,name:<name>” and the “,waitfor:<name>” modifiers.

Pre-Condition

In some cases we want the event to be fired only after a certain condition. For example: we want to start event to be fired when a function foo is called from the function bar. This can be done with the pre-condition event type. This event doesn’t actually calls to the tool (i.e. fires an event) but only arms the next alarm in the chain.

Example:

% sde -skx -mix -control precond:address:bar,start:address:foo,stop:icount:100000 -- myapp

The controller will arm the start event after calling to the bar function. Only now, when foo is called, the start event will be fired and the region start. The stop event will be triggered after 100000 instructions have been executed.

Repeating an alarm multiple times

By default the alarm is “armed” only once for each thread. Once the event is fired, this thread won’t trigger an additional event, for this alarm. If one adds the “,repeat:<int>” to the end of the alarm, it will be activated <int> times. A repeat token without an argument means that the alarm will be fired every time it is executed.

Example:

% sde -skx -mix -control start:address:foo,stop:icount:1000 -- myapp

The controller will fire a start event at the first call to the function foo, and the region will be terminated after 1000 instructions were executed. No more events will be fired even if the function foo is called again and again.

% sde -skx -mix -control start:address:foo,stop:icount:1000,repeat:3 -- myapp

In this example, the controller will fire a start event at the call to the function foo, and the region will be terminated after 1000 instructions were executed. Then it will “re-arm” the address alarm, and fire a start event again at the next call to the foo function. The region will be terminated again after additional 1000 instructions were executed. This is repeated 3 times. As opposed to using the “:count3” syntax, which will fire a single start event, only once the start condition was executed 3 time.

Each tool might act differently on repeating of the region. In most cases the analysis will aggregate across all the active regions.

When the repeat modifier doesn’t include the repeat count, the region is started and stopped every time the conditions are met.

% sde -skx -mix -control start:address:foo,stop:icount:1000,repeat -- myapp

In this example, the mix analysis tool will include the aggregation of all the regions starting from the call to the foo function and terminate after 1000 instructions.

Multi-threading

The alarm-chains are handled separately for each thread. The controller “arms” the <alarm-type> separately for each thread, and the alarm’s value is counted separately for each thread.

When the “:tid<int>” syntax is used, the preceding alarm is armed only for the thread with the input thread ID.

When the “:global” token is used, the alarm will be counted in all threads.

Adding the “:bcast” token, will cause the controller to specify that the event is marked with the broadcasting attribute (upon the arrival of the event). This means that when the alarm is executed, it fires the event to all active threads. The tool can use this information to decide if to profile all the threads based on this event or only the triggering thread. The effect of adding the “bcast” token depends on the way the tool handles this information. In Addition, an alarm defined with :bcast, will ‘arm’ the following alarm in the chain, for all threads.

Going back to the example above:

% sde -skx -mix -control start:icount:100 -- myapp

For each thread that reaches an icount of 100, the controller will fire a start event to the tool. The event will be delivered with the tid of the triggering thread. Adding the “:tid<int> token as in the following example:

% sde -skx -mix -control start:icount:100:tid3 -- myapp

Means that only when the application thread with pin thread ID 3 reach the icount of 100, the controller will fire the start event to the tool. Other application threads will not get the start event.

Control Syntax Summary

The full definition of an alarm chain:

-control <alarm-chain>[,repeat[:<int>]][,name:<name>][,waitfor:<name>]

alarm-chain ::= <alarm>[,<alarm-chain>]

alarm ::= <event>:<type>:<value>[:count<int>][:tid<int>][:bcast][:global]

event ::= start|stop|threadid|precond

type ::= icount|address|ssc|itext|isa-extension|isa-category|isa-class|int3|magic|pcontrol|enter_func|exit_func|img_load|img_enter|syscall|interactive

Values per alarm type:

Alarm

Value

icount

unsigned long int

instcount

unsigned long int

address

  • <name> [symbol/function name]

  • <name>+<offset> [image name + offset]

  • 0x<hex address> [pc address]

ssc

hex (w/o 0x): read more about SSC marks in Using SSC Marks

itext

hex: raw bytes of the instruction

int3

no argument (the int3 instruction is skipped)

isa-extension

string: the instruction’s extension as specified by XED

isa-category

string: the instruction’s category as specified by XED

isa-class

string: the instruction’s ICLASS as specified by XED

enter_func

string: function name without namespace and params

exit_func

string: function name without namespace and params

img_load

string: image name

img_enter

string: image name

cpuid

hex: CPUID leaf (value in eax) match input value

magic

int.int: the instruction XCHG ebx,ebx and input/output values are defined by the numbers

pcontrol

string: the second argument to the MPI_Pcontrol function called by the application

timeout

int: number of seconds

syscall

int: number of the system call

interactive

no argument (requires additional knob (see below)

Note

Using the address alarm with image and offset, the ‘+’ sign is the key to distinguish between a function name and an image name. The image name can be full path or only the base name of the image.

Interactive Controller

Using the interactive controller requires two windows: one to run sde with the application and the interactive controller, the second to specify the start/stop event by using the controller client. Here is an example:

Application:
%  <kit>/sde -hsw -mix -control start:interactive:bcast,stop:interactive:bcast \
   -interactive-file /tmp/myapp_ctrl -early-out -- myapp

Main process pid: 32564
    ** using file: /tmp/myapp_ctrl.32564
    ** listening to port: 34106
Controlller start event:
% <kit>/misc/sde_cntrl_client /tmp/myapp_ctrl.32564

Controlller stop event:
% <kit>/misc/sde_cntrl_client /tmp/myapp_ctrl.32564

Alarms accuracy

The controller will fire the event once the alarm reaches the triggering condition. The instruction count alarm specifically works in a basic block granularity. The event will be fired at the beginning of a basic-block, when the current icount + the number of instructions within the basic-block, exceed the value defined in the controller, for triggering the event.

Most of the location based events (e.g. address, ssc, and more) where modified to be more accurate and they are triggers in the specific instruction and not at the basic block level.

Note

Using the controller for specifying region of interest for tracing with pinplay has special handling. The start and stop events always act on all the threads. Due to implementation limitations there is a small delay between the exact instruction on which the event is fired and when it is actually start or stop the region.

Special Alarm - Uniform:

Uniform sampling is a way to collect periodic regions at constant length from a long running application. The uniform sampling controller can be triggered at the start of the process or at certain event. It is defined as uniform:<period>:<length>:<count>

period: number of instructions before next sampling starts.
length: number of instructions to sample.
count: number of samples.

The alarm can be used to define multiple regions based on instruction count.

The Uniform Alaram Tokens

The uniform sampling controller follow the same modifiers defined in the general controller options.

repeat[<int>]: number of iterations of the chain (when no number provided, execute till the end of the process).

count<int>: delay firing the event to only the Nth execution of the alarm, the counting happen for each thread unless global is also specified.

tid<int>: the thread to monitor, events on other threads are ignored.

bcast: inform the tool that the event should be processed for all threads (this behavior is tool specific).

name<string>: the name of the chain, other chains can wait for this chain to finish before it starts.

waitfor<string>: start the chain only after the chain with the specified name has finished.

global: Count alarms summary for all threads and not in a specific thread. this token can not appear with tid token.

Controller knobs

-control

Define control events and conditions

-controller_default_start

Add default START event [default 1]

-controller_log

Emit the controller’s events log [default 0]

-controller_olog

File name to emit controller’s event log [default controller-log.txt]

-controller_rep_nonstop

Do not stop on repeat string instructions [default 0]

-interactive_file

Name of file used for interactive communication (Linux)

-interactive-event

Name of event used for interactive communication (Windows)

Backward Compatibility

Most of the knobs from the old controller are still supported by the new controller. You are encouraged to use the new controller as it provides better control.

-length

Number of instructions to execute before stopping

-skip

number of instructions to skip

-start_address

Address and count to trigger a start

-start_category

Trigger a start on seeing an instruction from this XED ISA category

-start_extension

Trigger a start on seeing an instruction from this XED ISA extension

-start_int3

Trigger a start on seeing an ‘int 3’ instruction, requires a count argument

-start_itext

Trigger a start on seeing specified instruction bytes (max 15Bytes)

-start_ssc_mark

Trigger a start on seeing specified SSC marker, big-endian hex without an 0x prefix

-stop_address

Address and count to trigger a stop

-stop_category

Trigger a stop on seeing an instruction from this XED ISA category

-stop_extension

Trigger a stop on seeing an instruction from this XED ISA extension

-stop_int3

Trigger a stop on seeing an ‘int 3’ instruction, requires a count argument

-stop_itext

Trigger a stop on seeing specified instruction bytes (max 15Bytes)

-stop_ssc_mark

Trigger a stop on seeing specified SSC marker

Controller support in SDE tools

As mentioned above, the controller is a self contained component within SDE. All aforementioned details, is related to when and what events are fired by the controller. This section will discuss which SDE tools support using the controller, and how do they handle events fired by the controller.

All analysis tools behave similarly: The tool collects data per thread. Based on an arrival of a start event triggered by that thread. The tool will stop the collections of data for that thread, when a stop event arrives for that thread. If the tool receives an event with “bcast” as the tid, rather than the triggering tid, the tool will apply this event on all threads.

The effect of the events are effective right at the point of arrival.

Tracing Support

The tracing tool handles a global region. Meaning, at each given time, we are either in a region, and tracing all threads, or we are outside a region, and not tracing any thread. Another special behavior of the tracing tool is the transition from in/out a region. The effect of the event arrival isn’t immediate.

Following is a description of what happens in the tracing tool upon an arrival of a start event. When the thread which “caught” the event reaches the end of it’s current Basic-Block (BBL), it stops and calls all other threads to stop execution. Each thread is stopped at the end of it’s next BBL. Once all treads are stopped, Intel® SDE “removes” the current instrumentation and change the mode to “in-region”, meaning that all executing code will be instrumented with the trace generation instrumentation.

At this point, all threads are resumed. The same process is done for stopping the trace generation.

Note

The tracing tool ignores any event that arrive after recieving the event and resuming the execution (steps 1 and 4). The controller behavior is orthogonal to the way abalysis tools handle the events. So the controller will continue firing events based on the alarms defined by the user in the command line. The tracing tool will ignore them if a previous event arrived, and its processing hasn’t been completed yet.

By default, sde analysis tools follow the same behavior when running together with the tracing tool. You can cancel this by adding ‘-pinplay-control 0’, this will keep the behavior of the sde analysis tools as described above.