Running Applications with TSX

Intel Transactional Synchronization Extensions known as Intel TSX is an extension to the x86 ISA that adds hardware transactional memory support. It enables optimistic execution of transactional code regions. The hardware monitors multiple threads for conflicting memory accesses, while aborting and rolling back transactions that cannot be successfully completed. Mechanisms are provided for software to detect and handle failed transactions.

Restricted Transactional Memory (RTM) is an implementation of TSX which gives the programmer the flexibility to specify a fallback code path that is executed when a transaction cannot be successfully executed.

Intel® SDE supports running applications with TSX. Intel® SDE emulation monitors the memory operations in all the threads and detects conflicts and other conditions that may abort transaction. Intel® SDE allow debugging applications with TSX with its transparent application debugging, and can provide detailed statistics.

Intel® SDE TSX emulation adds significant runtime overhead, therefore users are required to instruct Intel® SDE to turn on TSX emulation with the -tsx knob and to specify the RTM emulation mode with the -rtm-mode knob.

Here is the list of options available when running applications with TSX.

-tsx

Enable TSX (RTM) functionality [default 0]

-rtm_mode

Select RTM mode from (options: disabled, abort, full, nop) [default disabled]

-rtm_abort_reason

Select RTM abort reason code, relevant only for RTM abort mode [default 0]

-rtm_extended_abort_code

Report extended abort cause codes [default 0]

-tsx_bloom_filter_type

Bloom filter CPU type : HSW, BDW, SNC [default BDW]

-tsx_cache_set_size

Number of cache lines in associative cache set [default 8]

-tsx_cache_sets_num

Number of cache sets in cache [default 64]

-tsx_debug_log

Debug log details level, printed to the the log file [default 0]

-tsx_file_name

TSX log file name [default sde-tsx-out.txt]

-tsx_log_file

Create a log file (does not necessarily fill it) [default 0]

-tsx_log_flush

Flush data to the log file after each write [default 0]

-tsx_log_inst

Print the instructions to the log file [default 0]

-tsx_ot_accuracy

Ownership table accuracy level : (options: simple, moderate) [default moderate]

-tsx_ownership_size

Ownership table size expressed in log2(ownership table number of entries) [default 16]

-tsx_speculation_depth

Maximum speculation depth allowed [default 7]

-tsx_stats

Collect TSX statistics [default 0]

-tsx_stats_call_stack

Add call stack information to TSX statistics [default 0]

-tsx_stats_call_stack_size

Call stack size in TSX statistics [default 10]

-tsx_stats_file

TSX statistics file name [default sde-tsx-stats.txt]

-tsx_stats_max_abort

Maximum number of TSX statistics abort list [default 1000000]

-tsx_stats_regs

Create register information to TSX statistics [default 0]

-tsx_stats_regs_file

TSX registers statistics file name [default sde-tsx-stats-regs.txt]

-tsx_stats_top_most

Number of most common aborts TSX statistics to display [default 10]

Examples for Running TSX Applications

% sde -bdw -rtm-mode full -- <application>

Running sde with full RTM support with the BDW CPU model.

% sde -bdw -tsx -tsx-stats -- <application>

Running sde with TSX support and statistics.

% sde -bdw -tsx -tsx-stats -tsx-stats-call-stack -- <application>

Running sde with TSX support and statistics and call stack information.

TSX Statistics Output

# LIST OF RTM COUNTERS DATA PER THREAD
#------------------------------------------------------------------
# TID   OS TID      XBEGIN        XEND      XABORT  GENERAL ABORTS  CONTENTION ABORTS
    0    21051           0           0           0               0                  0
    1    21442           5           4           0               0                  1
    2    21447         497         497           0               0                  0
    3    21448           1           1           0               0                  0
    4    21449           1           1           0               0                  0
    5    21450         317         316           0               0                  1
    6    21451         185         185           0               0                  0
    7    21452           1           1           0               0                  0
    8    21453           1           1           0               0                  0
    9    21454         187         186           0               0                  1
   10    21455         314         314           0               0                  0
   11    21456           3           2           0               0                  1
   12    21457           1           1           0               0                  0
   TOTAL              1513        1509           0               0                  4
# COUNTERS OF TSX ABORTS PER ABORT REASON
#------------------------------------------------------------------
# REASON                         RTM ABORTS   HLE ABORTS
ABORT_CONTENTION                          4            0
# TOP 10 CONTENTION ABORTS
#------------------------------------------------------------------
#        IP   COUNT    INSTRUCTION DISASSEMBLY
0x000401a60       3    mov eax, dword ptr [rip+0x2652]
0x00401110        1    jmp qword ptr [rip+0x2f72]
# STACK INFORMATION OF CONTENTION ABORT KILLER FOR IP: 0x00401a60
#------------------------------------------------------------------
#                IP           FUNCTION                                        IMAGE NAME
0# 0x00401a60      _ZNK19SharedSerialFibBodyIN8test_rtm9rtm_mutexEEclEm   fibonacci.exe:0x1a60
1# 0x00401a2c      _ZNK19SharedSerialFibBodyIN8test_rtm9rtm_mutexEEclEm   fibonacci.exe:0x1a2c
2# 0x00401ca4      _ZN21NativeParallelForTaskIi8test_rtm9rtm_mutexEEEEPv  fibonacci.exe:0x1ca4
3# 0x2aaaf279f955  start_thread                                           libpthread.so.0:0x8955
4# 0x2aaaf279f640  start_thread                                           libpthread.so.0:0x8640
#LIST OF TSX CONTENTION ABORT EVENTS
#------------------------------------------------------------------
# TID  VICTIM IP  KILLER TID KILLER IP   KILLER DATA ADDRESS  INSIDE TSX  LOCK ADDRESS  TSX TYPE
    1 0x004012ae           2 0x00401a60           0x004040b8         YES           N/A       RTM
    5 0x004012ae           0 0x00401110           0x00404088          NO           N/A       RTM
    9 0x004012ae          10 0x00401a60           0x004040b8         YES           N/A       RTM
   11 0x004012ae          10 0x00401a60           0x004040b8         YES           N/A       RTM

Example of a simple C TSX Application (Windows)

volatile int winning_thread = -1;
volatile int aborts = 0;
volatile int num_ends = 0;

unsigned __stdcall thread_worker(void * arg)
{
    int id =(int) arg;
    unsigned int status = _xbegin();
    if (status == _XBEGIN_STARTED)
    {
        for (int i=0; i<10000000; i++) winning_thread = id;
        num_ends++;
        _xend();
    }
    else
    {
        aborts++;
    }
    return 0;
}

int main()
{
    HANDLE threads[10];
    for (int i=0; i<10; i++)
        threads[i] = (HANDLE) _beginthreadex(NULL, 0, &thread_worker, (void *)i, 0, NULL);
    for (int i=0; i<10; i++)
        WaitForSingleObject( threads[i], INFINITE );
    return  0;
}