Tools/likwid/example_perfctr_stream
[[TOC]]
Example
likwid-perfctr
performance group MEM
on
benchmark stream
Build
stream
benchmark with Intel compilermodule purge module add compiler/intel/2022 icc -std=c11 -Ofast -xHost -ipo -qopenmp \ -o stream stream.c
Set up OpenMP environment
export OMP_NUM_THREADS=76 export KMP_AFFINITY=verbose,granularity=core,respect,scatter
List available performance groups
likwid-perfctr -a
... MEM Memory bandwidth in MBytes/s MEM_DP Overview of arithmetic and main memory performance MEM_FREERUN Memory bandwidth in MBytes/s MEM_SP Overview of arithmetic and main memory performance ...
Get detailed information on performance groups
likwid-perfctr -H --group MEM
Group MEM: Formulas: Memory read bandwidth [MBytes/s] = 1.0E-06*(SUM(CAS_COUNT_RD))*64.0/runtime Memory read data volume [GBytes] = 1.0E-09*(SUM(CAS_COUNT_RD))*64.0 Memory write bandwidth [MBytes/s] = 1.0E-06*(SUM(CAS_COUNT_WR))*64.0/runtime Memory write data volume [GBytes] = 1.0E-09*(SUM(CAS_COUNT_WR))*64.0 Memory bandwidth [MBytes/s] = 1.0E-06*(SUM(CAS_COUNT_RD)+SUM(CAS_COUNT_WR))*64.0/runtime Memory data volume [GBytes] = 1.0E-09*(SUM(CAS_COUNT_RD)+SUM(CAS_COUNT_WR))*64.0 - Profiling group to measure memory bandwidth drawn by all cores of a socket. Since this group is based on Uncore events it is only possible to measure on a per socket base. Some of the counters may not be available on your system. Also outputs total data volume transferred from main memory.
Measure performance group
MEM
for benchmarkstream
on CPU Hyperthread 0 to 151likwid-perfctr \ --group MEM \ -c 0-151 \ -n 1000000000 ./stream
------------------------------------------------------------- STREAM version $Revision: 5.10 $ ------------------------------------------------------------- This system uses 8 bytes per array element. ------------------------------------------------------------- Array size = 999999944 (elements) Memory per array = 7629.4 MiB (= 7.5 GiB). Total memory required = 22888.2 MiB (= 22.4 GiB). Each kernel will be executed 10 times. The *best* time for each kernel (excluding the first iteration) will be used to compute the reported bandwidth. ------------------------------------------------------------- OpenMP version (yyyymm): 201611 Number of Threads requested = 76 Number of Threads counted = 76 ------------------------------------------------------------- Your clock granularity appears to be 1000 ticks per microseconds. Each test below will take on the order of 50701 microseconds. (= 50701851 clock ticks) Increase the size of the arrays if this shows that you are not getting at least 20 clock ticks per test. ------------------------------------------------------------- WARNING -- The above is only a rough guideline. For best results, please be sure you know the precision of your system timer. ------------------------------------------------------------- Function Best Rate MB/s Med time Min time Max time Copy: 315666.4 0.050794 0.050686 0.052201 Scale: 313917.3 0.051067 0.050969 0.052823 Add: 319821.6 0.075599 0.075042 0.077726 Triad: 319194.3 0.076074 0.075189 0.077786 ------------------------------------------------------------- Solution Validates: avg error less than 1.000000e-13 on all three arrays -------------------------------------------------------------
-------------------------------------------------------------------------------- Group 1: MEM +-----------------------+---------+------------+-------------+------------+-------------+------------+-------------+-------------+------------+-------------+------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+ | Event | Counter | HWThread 0 | HWThread 1 | HWThread 2 | HWThread 3 | HWThread 4 | HWThread 5 | HWThread 6 | HWThread 7 | HWThread 8 | HWThread 9 | HWThread 10 | HWThread 11 | HWThread 12 | HWThread 13 | HWThread 14 | HWThread 15 | HWThread 16 | HWThread 17 | HWThread 18 | HWThread 19 | HWThread 20 | HWThread 21 | HWThread 22 | HWThread 23 | HWThread 24 | HWThread 25 | HWThread 26 | HWThread 27 | HWThread 28 | HWThread 29 | HWThread 30 | HWThread 31 | HWThread 32 | HWThread 33 | HWThread 34 | HWThread 35 | HWThread 36 | HWThread 37 | HWThread 38 | HWThread 39 | HWThread 40 | HWThread 41 | HWThread 42 | HWThread 43 | HWThread 44 | HWThread 45 | HWThread 46 | HWThread 47 | HWThread 48 | HWThread 49 | HWThread 50 | HWThread 51 | HWThread 52 | HWThread 53 | HWThread 54 | HWThread 55 | HWThread 56 | HWThread 57 | HWThread 58 | HWThread 59 | HWThread 60 | HWThread 61 | HWThread 62 | HWThread 63 | HWThread 64 | HWThread 65 | HWThread 66 | HWThread 67 | HWThread 68 | HWThread 69 | HWThread 70 | HWThread 71 | HWThread 72 | HWThread 73 | HWThread 74 | HWThread 75 | HWThread 76 | HWThread 77 | HWThread 78 | HWThread 79 | HWThread 80 | HWThread 81 | HWThread 82 | HWThread 83 | HWThread 84 | HWThread 85 | HWThread 86 | HWThread 87 | HWThread 88 | HWThread 89 | HWThread 90 | HWThread 91 | HWThread 92 | HWThread 93 | HWThread 94 | HWThread 95 | HWThread 96 | HWThread 97 | HWThread 98 | HWThread 99 | HWThread 100 | HWThread 101 | HWThread 102 | HWThread 103 | HWThread 104 | HWThread 105 | HWThread 106 | HWThread 107 | HWThread 108 | HWThread 109 | HWThread 110 | HWThread 111 | HWThread 112 | HWThread 113 | HWThread 114 | HWThread 115 | HWThread 116 | HWThread 117 | HWThread 118 | HWThread 119 | HWThread 120 | HWThread 121 | HWThread 122 | HWThread 123 | HWThread 124 | HWThread 125 | HWThread 126 | HWThread 127 | HWThread 128 | HWThread 129 | HWThread 130 | HWThread 131 | HWThread 132 | HWThread 133 | HWThread 134 | HWThread 135 | HWThread 136 | HWThread 137 | HWThread 138 | HWThread 139 | HWThread 140 | HWThread 141 | HWThread 142 | HWThread 143 | HWThread 144 | HWThread 145 | HWThread 146 | HWThread 147 | HWThread 148 | HWThread 149 | HWThread 150 | HWThread 151 | +-----------------------+---------+------------+-------------+------------+-------------+------------+-------------+-------------+------------+-------------+------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+ | INSTR_RETIRED_ANY | FIXC0 | 411100 | 1184249672 | 149965269 | 1104712055 | 5546 | 1134475482 | 1106776815 | 4585 | 1155439103 | 12296 | 4585 | 1133044286 | 7507 | 1175522960 | 4585 | 1176669204 | 7511 | 1077555593 | 1225750009 | 35137 | 1061558890 | 1124879267 | 7521 | 49848 | 1118060637 | 4585 | 95895 | 1172589157 | 49283 | 4585 | 339399 | 1130570920 | 106130 | 1064293858 | 95672 | 1080836037 | 1062857265 | 1041121006 | 300022 | 108731 | 5210 | 1096420996 | 5210 | 1161114225 | 5210 | 1165561806 | 5210 | 1115731717 | 5210 | 5210 | 1219127397 | 5210 | 5210 | 5210 | 5210 | 5210 | 5210 | 5210 | 5210 | 1109346629 | 1098478129 | 1228983442 | 1072362635 | 5210 | 5210 | 1253013370 | 1147521883 | 7578 | 5210 | 209643 | 1117508943 | 6378 | 1125844211 | 1111134889 | 1116539023 | 1046789695 | 980599496 | 89938 | 1041987456 | 52008 | 1064541406 | 31215 | 127663 | 1135079049 | 1221484 | 1225694490 | 1175780782 | 63646 | 1130007390 | 51915 | 1119630406 | 78499 | 1113596627 | 77069 | 180029 | 1099838645 | 305288 | 62306 | 1143319614 | 1279573502 | 206748 | 1157283145 | 1201574853 | 21244 | 1112756450 | 1109711139 | 1100917812 | 14453 | 1123402981 | 12931565 | 1105607959 | 73971 | 1078831 | 4585 | 1093321245 | 1153278368 | 1122217485 | 717997 | 1101821767 | 1449411 | 1142387553 | 427108 | 1168376524 | 9105 | 1128198161 | 1108818941 | 769770 | 1155300925 | 1088996221 | 1203010445 | 1099374333 | 1108141873 | 1213718559 | 1242045193 | 1183471981 | 25900 | 63971 | 5210 | 8150 | 1095593945 | 1124496587 | 1458043 | 18955 | 1133986401 | 1184529257 | 1124784030 | 30431 | 1196440890 | 4585 | 4585 | 4585 | 1948681 | | CPU_CLK_UNHALTED_CORE | FIXC1 | 2985530 | 9439091228 | 2038977682 | 9433147786 | 168923 | 9420946404 | 9416799758 | 22952 | 9416116346 | 589895 | 22209 | 9445960634 | 250101 | 9433697730 | 23363 | 9401933986 | 240074 | 9461707416 | 9434626883 | 1062792 | 9417078519 | 9425422162 | 199210 | 103127 | 9411006353 | 22239 | 137168 | 9463585824 | 100810 | 22907 | 508217 | 9435754518 | 829427 | 9425695784 | 1907666 | 9467246607 | 9433036222 | 9409644329 | 1897813 | 773580 | 29738 | 9481594756 | 29225 | 9448994798 | 28253 | 9480142033 | 28162 | 9414360456 | 29892 | 29128 | 9468913286 | 28334 | 29094 | 27947 | 27684 | 29122 | 27625 | 28631 | 28499 | 9474903346 | 9458976640 | 9452992986 | 9411305476 | 29709 | 28506 | 9483012327 | 9480183769 | 48274 | 32104 | 316403 | 9484033633 | 38067 | 9484289936 | 9487798043 | 9489652307 | 9457972619 | 9306451343 | 1420797 | 9473087354 | 257549 | 9441911027 | 557550 | 923374 | 9424417064 | 1331397 | 9423180587 | 9443671645 | 238582 | 9459289554 | 361990 | 9425127640 | 1563776 | 9415661725 | 1365579 | 3455936 | 9438274810 | 8611475 | 259780 | 9451412003 | 9457774235 | 3166896 | 9425002196 | 9448768860 | 837336 | 9408931178 | 9431467054 | 9415256076 | 173349 | 9440305505 | 15529130 | 9432717489 | 448507 | 4671565 | 17183 | 9470132532 | 9466522235 | 9485329999 | 4126153 | 9451790483 | 3080532 | 9455226635 | 1665879 | 9461947474 | 48294 | 9484261363 | 9433504069 | 2522473 | 9450491653 | 9438290472 | 9490319687 | 9462971849 | 9500151305 | 9469496066 | 9485594411 | 9499556637 | 234805 | 541356 | 43281 | 84040 | 9482517176 | 9444866240 | 42148972 | 182830 | 9470341710 | 9503596926 | 9481031469 | 302089 | 9501805133 | 28084 | 31161 | 33075 | 1014301 | | CPU_CLK_UNHALTED_REF | FIXC2 | 2263680 | 7190013696 | 1536292800 | 7184417760 | 128640 | 7176050016 | 7171745184 | 17376 | 7167935712 | 451200 | 16512 | 7195451232 | 187680 | 7182403392 | 17280 | 7161032928 | 185280 | 7204870848 | 7177992384 | 797952 | 7170927936 | 7176303744 | 149664 | 77568 | 7168710720 | 16416 | 103200 | 7207636128 | 75840 | 17088 | 382464 | 7190376192 | 638976 | 7179895008 | 1465824 | 7210287744 | 7184311584 | 7169324832 | 1467264 | 594720 | 22080 | 7219194048 | 21984 | 7195203168 | 21120 | 7216567392 | 21216 | 7167686400 | 22272 | 21792 | 7207089888 | 21216 | 21792 | 21120 | 20928 | 21312 | 20736 | 21312 | 21216 | 7215043488 | 7202249856 | 7198327584 | 7166597376 | 22080 | 21984 | 7214701344 | 7216876224 | 36288 | 24000 | 237504 | 7222905312 | 28512 | 7223367264 | 7222639392 | 7224492864 | 7204625376 | 7078540608 | 1081248 | 7195825536 | 192480 | 7192735392 | 429600 | 698304 | 7174492800 | 999456 | 7174404768 | 7190069280 | 182112 | 7203154656 | 278016 | 7176823680 | 1195968 | 7174627008 | 1035072 | 2663520 | 7188087744 | 6539712 | 195552 | 7193985408 | 7199400672 | 2432640 | 7175360736 | 7193941728 | 638784 | 7165195296 | 7185431520 | 7172219136 | 133152 | 7192677600 | 11955744 | 7186553952 | 346944 | 3616416 | 12576 | 7210151712 | 7202959104 | 7220774688 | 3096768 | 7196345760 | 2310624 | 7198027008 | 1249920 | 7203082464 | 36288 | 7221995040 | 7185022656 | 1898784 | 7191787488 | 7189887648 | 7222777920 | 7205333088 | 7235373600 | 7205773056 | 7219077120 | 7230773856 | 175104 | 406368 | 32448 | 63360 | 7218636480 | 7187362752 | 31611840 | 136992 | 7209568224 | 7234794816 | 7218941952 | 227712 | 7232235360 | 21024 | 23040 | 24960 | 760608 | | TOPDOWN_SLOTS | FIXC3 | 10657535 | 47172282470 | 5156016260 | 47153531225 | 618625 | 47079200600 | 47062350095 | 114720 | 47064241435 | 2214070 | 111005 | 47195236705 | 887450 | 47163416655 | 116775 | 46969367310 | 815850 | 47290145735 | 47152326120 | 4599810 | 47056566960 | 47118950500 | 679675 | 509485 | 47021320350 | 111135 | 685780 | 47301784005 | 442015 | 114475 | 2541025 | 47172984895 | 3531475 | 47022378985 | 7305915 | 47326297765 | 47122181610 | 47045606645 | 5811890 | 2743625 | 148630 | 47305311815 | 146065 | 47243578325 | 141205 | 47399169370 | 140750 | 47066250755 | 149400 | 145580 | 47311193325 | 141610 | 145410 | 139675 | 138360 | 145550 | 138065 | 143095 | 142435 | 47355442915 | 47277963115 | 47263584965 | 47047316745 | 148485 | 142470 | 47302550930 | 47387108905 | 217400 | 135780 | 1557295 | 47357475385 | 178685 | 47419190865 | 47435970790 | 47443919340 | 47285562040 | 43290869680 | 5687265 | 42083406485 | 1108990 | 47178652075 | 2191680 | 3914745 | 47121235435 | 5854065 | 47105635955 | 47217492775 | 1131755 | 47293755130 | 1405985 | 47114703285 | 5708710 | 47074319880 | 4519795 | 10095790 | 47170724315 | 22208520 | 1116285 | 47254662345 | 47286806005 | 11082090 | 47120899235 | 47241219250 | 3402955 | 47039865315 | 47155073870 | 47053458500 | 714055 | 47191468560 | 56241780 | 47117829235 | 1613845 | 13468760 | 64205 | 47325546785 | 47318028205 | 47425466705 | 18365745 | 47257792265 | 15350590 | 47275153555 | 8283625 | 47308820560 | 172035 | 47420348960 | 47125347560 | 11838640 | 47251463775 | 47190533730 | 47450786165 | 47313802010 | 47499783555 | 47346673210 | 47427175440 | 47496780050 | 831270 | 1693785 | 169480 | 308670 | 47411487795 | 47223515205 | 102240095 | 687885 | 47342567250 | 47515962200 | 47393797570 | 950850 | 47505369965 | 109445 | 117185 | 133300 | 4998585 | | CAS_COUNT_RD | MBOX0C0 | 565487546 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 565664152 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | | CAS_COUNT_WR | MBOX0C1 | 394832802 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 395297060 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | | CAS_COUNT_RD | MBOX1C0 | 565673329 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 565695080 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | | CAS_COUNT_WR | MBOX1C1 | 394772844 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 395358301 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | | CAS_COUNT_RD | MBOX2C0 | 565520834 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 565702248 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | | CAS_COUNT_WR | MBOX2C1 | 394661003 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 395129483 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | | CAS_COUNT_RD | MBOX3C0 | 565668266 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 565742668 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | | CAS_COUNT_WR | MBOX3C1 | 394751578 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 395202355 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | | CAS_COUNT_RD | MBOX4C0 | 565524973 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 565570895 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | | CAS_COUNT_WR | MBOX4C1 | 394643153 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 395041115 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | | CAS_COUNT_RD | MBOX5C0 | 565691821 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 565944656 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | | CAS_COUNT_WR | MBOX5C1 | 394667249 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 395302587 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | | CAS_COUNT_RD | MBOX6C0 | 565545031 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 565590202 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | | CAS_COUNT_WR | MBOX6C1 | 394475734 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 395059268 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | | CAS_COUNT_RD | MBOX7C0 | 565684926 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 565874152 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | | CAS_COUNT_WR | MBOX7C1 | 394625582 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 395220417 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | +-----------------------+---------+------------+-------------+------------+-------------+------------+-------------+-------------+------------+-------------+------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+ +----------------------------+---------+---------------+-------+-------------+--------------+ | Event | Counter | Sum | Min | Max | Avg | +----------------------------+---------+---------------+-------+-------------+--------------+ | INSTR_RETIRED_ANY STAT | FIXC0 | 86285205997 | 4585 | 1279573502 | 5.676658e+08 | | CPU_CLK_UNHALTED_CORE STAT | FIXC1 | 720357096927 | 17183 | 9503596926 | 4.739191e+09 | | CPU_CLK_UNHALTED_REF STAT | FIXC2 | 548464184352 | 12576 | 7235373600 | 3.608317e+09 | | TOPDOWN_SLOTS STAT | FIXC3 | 3586852836495 | 64205 | 47515962200 | 2.359772e+10 | | CAS_COUNT_RD STAT | MBOX0C0 | 1131151698 | 0 | 565664152 | 7.441787e+06 | | CAS_COUNT_WR STAT | MBOX0C1 | 790129862 | 0 | 395297060 | 5.198223e+06 | | CAS_COUNT_RD STAT | MBOX1C0 | 1131368409 | 0 | 565695080 | 7.443213e+06 | | CAS_COUNT_WR STAT | MBOX1C1 | 790131145 | 0 | 395358301 | 5.198231e+06 | | CAS_COUNT_RD STAT | MBOX2C0 | 1131223082 | 0 | 565702248 | 7.442257e+06 | | CAS_COUNT_WR STAT | MBOX2C1 | 789790486 | 0 | 395129483 | 5.195990e+06 | | CAS_COUNT_RD STAT | MBOX3C0 | 1131410934 | 0 | 565742668 | 7.443493e+06 | | CAS_COUNT_WR STAT | MBOX3C1 | 789953933 | 0 | 395202355 | 5.197065e+06 | | CAS_COUNT_RD STAT | MBOX4C0 | 1131095868 | 0 | 565570895 | 7.441420e+06 | | CAS_COUNT_WR STAT | MBOX4C1 | 789684268 | 0 | 395041115 | 5.195291e+06 | | CAS_COUNT_RD STAT | MBOX5C0 | 1131636477 | 0 | 565944656 | 7.444977e+06 | | CAS_COUNT_WR STAT | MBOX5C1 | 789969836 | 0 | 395302587 | 5.197170e+06 | | CAS_COUNT_RD STAT | MBOX6C0 | 1131135233 | 0 | 565590202 | 7.441679e+06 | | CAS_COUNT_WR STAT | MBOX6C1 | 789535002 | 0 | 395059268 | 5.194309e+06 | | CAS_COUNT_RD STAT | MBOX7C0 | 1131559078 | 0 | 565874152 | 7.444468e+06 | | CAS_COUNT_WR STAT | MBOX7C1 | 789845999 | 0 | 395220417 | 5.196355e+06 | +----------------------------+---------+---------------+-------+-------------+--------------+ +-----------------------------------+-------------+------------+------------+------------+------------+------------+------------+--------------+------------+------------+--------------+-------------+-------------+-------------+--------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+--------------+-------------+--------------+-------------+-------------+--------------+--------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+--------------+-------------+--------------+-------------+--------------+-------------+--------------+-------------+--------------+--------------+-------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+-------------+-------------+-------------+-------------+--------------+--------------+-------------+-------------+--------------+--------------+-------------+-------------+--------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+ | Metric | HWThread 0 | HWThread 1 | HWThread 2 | HWThread 3 | HWThread 4 | HWThread 5 | HWThread 6 | HWThread 7 | HWThread 8 | HWThread 9 | HWThread 10 | HWThread 11 | HWThread 12 | HWThread 13 | HWThread 14 | HWThread 15 | HWThread 16 | HWThread 17 | HWThread 18 | HWThread 19 | HWThread 20 | HWThread 21 | HWThread 22 | HWThread 23 | HWThread 24 | HWThread 25 | HWThread 26 | HWThread 27 | HWThread 28 | HWThread 29 | HWThread 30 | HWThread 31 | HWThread 32 | HWThread 33 | HWThread 34 | HWThread 35 | HWThread 36 | HWThread 37 | HWThread 38 | HWThread 39 | HWThread 40 | HWThread 41 | HWThread 42 | HWThread 43 | HWThread 44 | HWThread 45 | HWThread 46 | HWThread 47 | HWThread 48 | HWThread 49 | HWThread 50 | HWThread 51 | HWThread 52 | HWThread 53 | HWThread 54 | HWThread 55 | HWThread 56 | HWThread 57 | HWThread 58 | HWThread 59 | HWThread 60 | HWThread 61 | HWThread 62 | HWThread 63 | HWThread 64 | HWThread 65 | HWThread 66 | HWThread 67 | HWThread 68 | HWThread 69 | HWThread 70 | HWThread 71 | HWThread 72 | HWThread 73 | HWThread 74 | HWThread 75 | HWThread 76 | HWThread 77 | HWThread 78 | HWThread 79 | HWThread 80 | HWThread 81 | HWThread 82 | HWThread 83 | HWThread 84 | HWThread 85 | HWThread 86 | HWThread 87 | HWThread 88 | HWThread 89 | HWThread 90 | HWThread 91 | HWThread 92 | HWThread 93 | HWThread 94 | HWThread 95 | HWThread 96 | HWThread 97 | HWThread 98 | HWThread 99 | HWThread 100 | HWThread 101 | HWThread 102 | HWThread 103 | HWThread 104 | HWThread 105 | HWThread 106 | HWThread 107 | HWThread 108 | HWThread 109 | HWThread 110 | HWThread 111 | HWThread 112 | HWThread 113 | HWThread 114 | HWThread 115 | HWThread 116 | HWThread 117 | HWThread 118 | HWThread 119 | HWThread 120 | HWThread 121 | HWThread 122 | HWThread 123 | HWThread 124 | HWThread 125 | HWThread 126 | HWThread 127 | HWThread 128 | HWThread 129 | HWThread 130 | HWThread 131 | HWThread 132 | HWThread 133 | HWThread 134 | HWThread 135 | HWThread 136 | HWThread 137 | HWThread 138 | HWThread 139 | HWThread 140 | HWThread 141 | HWThread 142 | HWThread 143 | HWThread 144 | HWThread 145 | HWThread 146 | HWThread 147 | HWThread 148 | HWThread 149 | HWThread 150 | HWThread 151 | +-----------------------------------+-------------+------------+------------+------------+------------+------------+------------+--------------+------------+------------+--------------+-------------+-------------+-------------+--------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+--------------+-------------+--------------+-------------+-------------+--------------+--------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+--------------+-------------+--------------+-------------+--------------+-------------+--------------+-------------+--------------+--------------+-------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+-------------+-------------+-------------+-------------+--------------+--------------+-------------+-------------+--------------+--------------+-------------+-------------+--------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+ | Runtime (RDTSC) [s] | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | 3.1710 | | Runtime unhalted [s] | 0.0012 | 3.9422 | 0.8516 | 3.9397 | 0.0001 | 3.9346 | 3.9329 | 9.585834e-06 | 3.9326 | 0.0002 | 9.275522e-06 | 3.9451 | 0.0001 | 3.9400 | 9.757487e-06 | 3.9267 | 0.0001 | 3.9517 | 3.9403 | 0.0004 | 3.9330 | 3.9365 | 0.0001 | 4.307068e-05 | 3.9305 | 9.288051e-06 | 0.0001 | 3.9524 | 4.210299e-05 | 9.567039e-06 | 0.0002 | 3.9408 | 0.0003 | 3.9366 | 0.0008 | 3.9540 | 3.9397 | 3.9299 | 0.0008 | 0.0003 | 1.241999e-05 | 3.9600 | 1.220573e-05 | 3.9463 | 1.179978e-05 | 3.9594 | 1.176177e-05 | 3.9319 | 1.248430e-05 | 1.216522e-05 | 3.9547 | 1.183361e-05 | 1.215102e-05 | 1.167198e-05 | 1.156214e-05 | 1.216272e-05 | 1.153750e-05 | 1.195765e-05 | 1.190252e-05 | 3.9572 | 3.9505 | 3.9480 | 3.9306 | 1.240787e-05 | 1.190544e-05 | 3.9606 | 3.9594 | 2.016149e-05 | 1.340814e-05 | 0.0001 | 3.9610 | 1.589857e-05 | 3.9611 | 3.9626 | 3.9633 | 3.9501 | 3.8868 | 0.0006 | 3.9564 | 0.0001 | 3.9434 | 0.0002 | 0.0004 | 3.9361 | 0.0006 | 3.9356 | 3.9441 | 0.0001 | 3.9506 | 0.0002 | 3.9364 | 0.0007 | 3.9324 | 0.0006 | 0.0014 | 3.9419 | 0.0036 | 0.0001 | 3.9474 | 3.9500 | 0.0013 | 3.9363 | 3.9462 | 0.0003 | 3.9296 | 3.9390 | 3.9323 | 0.0001 | 3.9427 | 0.0065 | 3.9395 | 0.0002 | 0.0020 | 7.176428e-06 | 3.9552 | 3.9537 | 3.9615 | 0.0017 | 3.9475 | 0.0013 | 3.9489 | 0.0007 | 3.9518 | 2.016984e-05 | 3.9611 | 3.9399 | 0.0011 | 3.9470 | 3.9419 | 3.9636 | 3.9522 | 3.9677 | 3.9549 | 3.9616 | 3.9675 | 0.0001 | 0.0002 | 1.807618e-05 | 3.509905e-05 | 3.9603 | 3.9446 | 0.0176 | 0.0001 | 3.9553 | 3.9691 | 3.9597 | 0.0001 | 3.9684 | 1.172920e-05 | 1.301430e-05 | 1.381367e-05 | 0.0004 | | Clock [MHz] | 3157.8904 | 3143.3382 | 3177.8188 | 3143.8058 | 3144.1511 | 3143.4006 | 3143.9030 | 3162.7246 | 3145.3455 | 3130.3743 | 3220.4753 | 3143.2487 | 3190.7155 | 3144.8708 | 3237.2446 | 3143.6354 | 3102.4675 | 3144.3723 | 3147.1134 | 3189.0561 | 3144.3544 | 3144.7828 | 3187.0174 | 3183.3210 | 3143.2988 | 3243.6842 | 3182.4659 | 3143.7899 | 3182.7017 | 3209.7236 | 3181.6271 | 3142.0687 | 3108.0233 | 3143.3010 | 3116.0984 | 3143.8495 | 3143.8151 | 3142.5746 | 3096.9615 | 3114.4642 | 3224.8041 | 3144.7297 | 3183.0133 | 3144.3668 | 3203.0322 | 3145.3923 | 3178.2689 | 3144.8684 | 3213.5599 | 3200.3997 | 3145.7981 | 3197.6802 | 3196.6640 | 3168.3411 | 3167.3187 | 3271.8067 | 3189.8330 | 3216.6437 | 3216.3016 | 3144.3182 | 3144.6088 | 3144.3319 | 3144.3256 | 3221.6593 | 3104.7041 | 3147.1584 | 3145.2716 | 3185.2308 | 3202.8644 | 3189.7769 | 3143.9224 | 3196.7717 | 3143.8063 | 3145.2861 | 3145.0937 | 3143.2382 | 3147.9732 | 3146.2800 | 3152.1115 | 3203.7964 | 3143.0875 | 3107.4933 | 3166.0937 | 3145.2411 | 3189.5877 | 3144.8670 | 3144.8393 | 3136.8212 | 3144.3178 | 3117.5787 | 3144.4567 | 3130.7301 | 3142.2604 | 3158.9076 | 3106.7076 | 3143.9085 | 3152.8955 | 3180.7834 | 3145.7035 | 3145.4533 | 3117.0704 | 3145.0559 | 3144.8429 | 3138.6030 | 3144.1475 | 3142.8022 | 3143.1799 | 3117.1973 | 3142.5783 | 3110.0056 | 3142.7279 | 3095.2839 | 3092.9626 | 3271.5014 | 3144.8671 | 3146.8074 | 3145.2799 | 3190.2690 | 3144.7977 | 3192.1780 | 3145.2062 | 3191.1843 | 3145.2328 | 3186.5504 | 3144.3941 | 3143.6599 | 3180.8385 | 3146.3585 | 3143.1267 | 3146.0617 | 3144.5908 | 3143.8384 | 3146.5667 | 3146.1072 | 3145.6414 | 3210.7162 | 3189.7313 | 3193.7433 | 3175.8613 | 3145.2786 | 3146.4215 | 3192.4776 | 3195.5300 | 3145.1911 | 3145.2302 | 3144.6527 | 3176.4326 | 3145.7501 | 3198.4110 | 3238.3185 | 3172.8236 | 3192.9831 | | CPI | 7.2623 | 7.9705 | 13.5963 | 8.5390 | 30.4585 | 8.3042 | 8.5083 | 5.0059 | 8.1494 | 47.9745 | 4.8438 | 8.3368 | 33.3157 | 8.0251 | 5.0955 | 7.9903 | 31.9630 | 8.7807 | 7.6970 | 30.2471 | 8.8710 | 8.3791 | 26.4872 | 2.0688 | 8.4173 | 4.8504 | 1.4304 | 8.0707 | 2.0455 | 4.9961 | 1.4974 | 8.3460 | 7.8152 | 8.8563 | 19.9396 | 8.7592 | 8.8752 | 9.0380 | 6.3256 | 7.1146 | 5.7079 | 8.6478 | 5.6094 | 8.1379 | 5.4228 | 8.1335 | 5.4054 | 8.4378 | 5.7374 | 5.5908 | 7.7670 | 5.4384 | 5.5843 | 5.3641 | 5.3136 | 5.5896 | 5.3023 | 5.4954 | 5.4701 | 8.5410 | 8.6110 | 7.6917 | 8.7762 | 5.7023 | 5.4714 | 7.5682 | 8.2614 | 6.3703 | 6.1620 | 1.5092 | 8.4868 | 5.9685 | 8.4242 | 8.5388 | 8.4992 | 9.0352 | 9.4906 | 15.7975 | 9.0914 | 4.9521 | 8.8695 | 17.8616 | 7.2329 | 8.3029 | 1.0900 | 7.6880 | 8.0318 | 3.7486 | 8.3710 | 6.9727 | 8.4181 | 19.9210 | 8.4552 | 17.7189 | 19.1966 | 8.5815 | 28.2077 | 4.1694 | 8.2666 | 7.3913 | 15.3177 | 8.1441 | 7.8637 | 39.4152 | 8.4555 | 8.4990 | 8.5522 | 11.9940 | 8.4033 | 1.2009 | 8.5317 | 6.0633 | 4.3302 | 3.7477 | 8.6618 | 8.2084 | 8.4523 | 5.7468 | 8.5783 | 2.1254 | 8.2767 | 3.9004 | 8.0984 | 5.3041 | 8.4066 | 8.5077 | 3.2769 | 8.1801 | 8.6670 | 7.8888 | 8.6076 | 8.5730 | 7.8021 | 7.6371 | 8.0269 | 9.0658 | 8.4625 | 8.3073 | 10.3117 | 8.6551 | 8.3992 | 28.9079 | 9.6455 | 8.3514 | 8.0231 | 8.4292 | 9.9270 | 7.9417 | 6.1252 | 6.7963 | 7.2137 | 0.5205 | | Memory read bandwidth [MBytes/s] | 91322.3211 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 91342.2479 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | | Memory read data volume [GBytes] | 289.5870 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 289.6502 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | | Memory write bandwidth [MBytes/s] | 63725.2563 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 63809.6327 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | | Memory write data volume [GBytes] | 202.0755 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 202.3431 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | | Memory bandwidth [MBytes/s] | 155047.5774 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 155151.8806 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | | Memory data volume [GBytes] | 491.6625 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 491.9933 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | +-----------------------------------+-------------+------------+------------+------------+------------+------------+------------+--------------+------------+------------+--------------+-------------+-------------+-------------+--------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+--------------+-------------+--------------+-------------+-------------+--------------+--------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+--------------+-------------+--------------+-------------+--------------+-------------+--------------+-------------+--------------+--------------+-------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+-------------+-------------+-------------+-------------+--------------+--------------+-------------+-------------+--------------+--------------+-------------+-------------+--------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+ +----------------------------------------+-------------+--------------+-------------+-----------+ | Metric | Sum | Min | Max | Avg | +----------------------------------------+-------------+--------------+-------------+-----------+ | Runtime (RDTSC) [s] STAT | 481.9920 | 3.1710 | 3.1710 | 3.1710 | | Runtime unhalted [s] STAT | 300.8552 | 7.176428e-06 | 3.9691 | 1.9793 | | Clock [MHz] STAT | 480262.0290 | 3092.9626 | 3271.8067 | 3159.6186 | | CPI STAT | 1392.3753 | 0.5205 | 47.9745 | 9.1604 | | Memory read bandwidth [MBytes/s] STAT | 182664.5690 | 0 | 91342.2479 | 1201.7406 | | Memory read data volume [GBytes] STAT | 579.2372 | 0 | 289.6502 | 3.8108 | | Memory write bandwidth [MBytes/s] STAT | 127534.8890 | 0 | 63809.6327 | 839.0453 | | Memory write data volume [GBytes] STAT | 404.4186 | 0 | 202.3431 | 2.6606 | | Memory bandwidth [MBytes/s] STAT | 310199.4580 | 0 | 155151.8806 | 2040.7859 | | Memory data volume [GBytes] STAT | 983.6558 | 0 | 491.9933 | 6.4714 | +----------------------------------------+-------------+--------------+-------------+-----------+
- All memory related performance counters are only accounted on first CPU core on the socket
Validity check
Socket 0: Memory read bandwidth: 91322.3211 MBytes/s Memory write bandwidth: 63725.2563 MBytes/s + -------------------- 155047.5774 MBytes/s Memory bandwidth: 155047.5774 MBytes/s
Memory write data volume socket 0: 202.0755 GBytes Memory write data volume socket 1: 202.3431 GBytes + --------------- 404.4186 GBytes Memory write data volume [GBytes] STAT: 404.4186 GBytes
Memory read data volume socket 0: 289.5870 GBytes Memory read data volume socket 1: 289.6502 GBytes + ---------- 579.2372 GBytes Memory read data volume [GBytes] STAT: 579.2372 GBytes
#Elements/vec = 1,000,000,000 #Bytes/Element = 8 #Bytes/vec = 1,000,000,000 * 8 = 8,000,000,000 #GBytes/vec = 8,000,000,000 / 1,000,000,000 = 8 #Num repetition = 10 #Num worm up rounds = 1 #Num iterations = Num repetition + Num worm up rounds = 11 Copy: 1 vec. read, 1 vec. write Scale: 1 vec. read, 1 vec. write Add: 2 vec. read, 1 vec. write Triad: 2 vec. read, 1 vec. write Write: ------ * vec initialization: 3 vec. write * 8 GBytes/vec = 24 GB * timer initialization: 1 vec. write * 8 GBytes/vec = 8 GB * iteration: 4 vec. write * 11 iterations * 8 GBytes/vec = 352 GB -> Sum: 384 GB ~ 404.4186 Memory write data volume [GBytes] STAT Read: ----- * timer initialization: 1 vec. read * 8 GBytes/vec = 8 GB * iteration: 6 vec. read * 11 iterations * 8 GBytes/vec = 528 GB * validation: 3 vec. read * 8 GBytes/vec = 24 GB -> Sum: 560 GB ~ 579.2372 Memory read data volume [GBytes] STAT
Example
likwid-perfctr
performance group UPI
on
benchmark stream
Build
stream
benchmark with Intel compilermodule purge module add compiler/intel/2022 icc -std=c11 -Ofast -xHost -ipo -qopenmp \ -o stream stream.c
Set up OpenMP environment
export OMP_NUM_THREADS=76 export KMP_AFFINITY=verbose,granularity=core,respect,scatter
List available performance groups
likwid-perfctr -a
... UPI UPI data traffic ...
Get detailed information on performance groups
likwid-perfctr -H --group UPI
Group UPI: Formulas: Received data bandwidth [MByte/s] = 1.0E-06*(SUM(RXL_FLITS_ALL_DATA)/9.0)*64.0/runtime Received data volume [GByte] = 1.0E-09*(SUM(RXL_FLITS_ALL_DATA)/9.0)*64.0 Sent data bandwidth [MByte/s] = 1.0E-06*(SUM(TXL_FLITS_ALL_DATA)/9.0)*64.0/time Sent data volume [GByte] = 1.0E-09*(SUM(TXL_FLITS_ALL_DATA)/9.0)*64.0 Total data bandwidth [MByte/s] = 1.0E-06*((SUM(RXL_FLITS_ALL_DATA)+SUM(TXL_FLITS_ALL_DATA))/9.0)*64.0/time Total data volume [GByte] = 1.0E-09*((SUM(RXL_FLITS_ALL_DATA)+SUM(TXL_FLITS_ALL_DATA))/9.0)*64.0 -- This group measures the data traffic on the UPI (socket interconnect). The group measures all filled data slots (9 slots per 64 Byte data transfer), that's why the count needs to be divided by 9. These 9 data chunks are not transferred in a single flit but there is one flit for the header and three flits for the data. The metrics show higher values as expected because the events count also different transfers which include data.
Measure performance group
UPI
for benchmarkstream
on CPU Hyperthread 0 to 151 with locally allocated memorylikwid-perfctr --group UPI -c 0-151 \ --localalloc \ numactl -n 1000000000 ./stream
... ------------------------------------------------------------- Function Best Rate MB/s Med time Min time Max time Copy: 315404.1 0.050860 0.050729 0.052256 Scale: 313710.9 0.051093 0.051002 0.052351 Add: 319827.1 0.075225 0.075041 0.077298 Triad: 317870.4 0.075667 0.075502 0.077569 -------------------------------------------------------------
... +----------------------------------------+-------------+-----------+-----------+-----------+ | Metric | Sum | Min | Max | Avg | +----------------------------------------+-------------+-----------+-----------+-----------+ | Runtime (RDTSC) [s] STAT | 489.0904 | 3.2177 | 3.2177 | 3.2177 | | Runtime unhalted [s] STAT | 300.4339 | 0 | 3.9619 | 1.9765 | | Clock [MHz] STAT | 476654.1148 | 3094.0504 | 3257.4571 | 3135.8823 | | CPI STAT | 2147.5606 | 0.4915 | 202.9889 | 14.1287 | | Received data bandwidth [MByte/s] STAT | 321.8206 | 0 | 181.1360 | 2.1172 | | Received data volume [GByte] STAT | 1.0355 | 0 | 0.5828 | 0.0068 | | Sent data bandwidth [MByte/s] STAT | 321.8082 | 0 | 181.1440 | 2.1172 | | Sent data volume [GByte] STAT | 1.0355 | 0 | 0.5829 | 0.0068 | | Total data bandwidth [MByte/s] STAT | 643.6286 | 0 | 321.8285 | 4.2344 | | Total data volume [GByte] STAT | 2.0710 | 0 | 1.0355 | 0.0136 | +----------------------------------------+-------------+-----------+-----------+-----------+
- Total UPI data volume and bandwidth are very low => low inter socket traffic
Measure performance group
UPI
for benchmarkstream
on CPU Hyperthread 0 to 151 with all allocated memory in NUMA domain 0likwid-perfctr --group UPI -c 0-151 \ --membind=0 \ numactl -n 1000000000 ./stream
... ------------------------------------------------------------- Function Best Rate MB/s Med time Min time Max time Copy: 112951.9 0.141838 0.141653 0.141931 Scale: 112253.1 0.142650 0.142535 0.142793 Add: 114055.6 0.210627 0.210424 0.210888 Triad: 114081.4 0.210572 0.210376 0.210740 -------------------------------------------------------------
... +----------------------------------------+-------------+--------------+------------+-----------+ | Metric | Sum | Min | Max | Avg | +----------------------------------------+-------------+--------------+------------+-----------+ | Runtime (RDTSC) [s] STAT | 1333.6328 | 8.7739 | 8.7739 | 8.7739 | | Runtime unhalted [s] STAT | 834.0245 | 2.164660e-06 | 10.9996 | 5.4870 | | Clock [MHz] STAT | 481297.2958 | 3093.2066 | 3247.5981 | 3166.4296 | | CPI STAT | 3052.8227 | 0.4719 | 244.3819 | 20.0844 | | Received data bandwidth [MByte/s] STAT | 56071.5598 | 0 | 33598.6098 | 368.8918 | | Received data volume [GByte] STAT | 491.9660 | 0 | 294.7907 | 3.2366 | | Sent data bandwidth [MByte/s] STAT | 56071.5652 | 0 | 33598.5985 | 368.8919 | | Sent data volume [GByte] STAT | 491.9661 | 0 | 294.7906 | 3.2366 | | Total data bandwidth [MByte/s] STAT | 112143.1250 | 0 | 56071.5765 | 737.7837 | | Total data volume [GByte] STAT | 983.9321 | 0 | 491.9662 | 6.4732 | +----------------------------------------+-------------+--------------+------------+-----------+
- Total UPI data volume and bandwidth are very high => high inter socket traffic
- Memory bandwidth halved