mbox series

[mm-unstable,v1,0/8] mm: multi-gen LRU: memcg LRU

Message ID 20221201223923.873696-1-yuzhao@google.com (mailing list archive)
Headers show
Series mm: multi-gen LRU: memcg LRU | expand

Message

Yu Zhao Dec. 1, 2022, 10:39 p.m. UTC 
An memcg LRU is a per-node LRU of memcgs. It is also an LRU of LRUs,
since each node and memcg combination has an LRU of folios (see
mem_cgroup_lruvec()).

Its goal is to improve the scalability of global reclaim, which is
critical to systemwide memory overcommit in data centers. Note that
memcg reclaim is currently out of scope.

Its memory bloat is a pointer to each LRU vector and negligible to
each node. In terms of traversing memcgs during global reclaim, it
improves the best-case complexity from O(n) to O(1) and does not
affect the worst-case complexity O(n). Therefore, on average, it has
a sublinear complexity in contrast to the current linear complexity.

The basic structure of an memcg LRU can be understood by an analogy
to the active/inactive LRU (of folios):
1. It has the young and the old (generations);
2. Its linked lists have the head and the tail;
3. The increment of max_seq triggers promotion;
4. Other events, e.g., offlining an memcg, triggers similar
   operations.

In terms of global reclaim, it has two distinct features:
1. Sharding, which allows each thread to start at a random memcg (in
   the old generation) and improves parallelism;
2. Eventual fairness, which allows direct reclaim to bail out and
   reduces latency without affecting fairness over some time.

The commit message in patch 6 details the workflow:
https://lore.kernel.org/r/20221201223923.873696-7-yuzhao@google.com/

The following is a simple test to quickly verify its effectiveness.
More benchmarks are coming soon.

  Test design:
  1. Create multiple memcgs.
  2. Each memcg contains a job (fio).
  3. All jobs access the same amount of memory randomly.
  4. The system does not experience global memory pressure.
  5. Periodically write to the root memory.reclaim.

  Desired outcome:
  1. All memcgs have similar pgsteal, i.e.,
     stddev(pgsteal)/mean(pgsteal) is close to 0%.
  2. The total pgsteal is close to the total requested through
     memory.reclaim, i.e., sum(pgsteal)/sum(requested) is close to
     100%.

  Actual outcome [1]:
             stddev(pgsteal)/mean(pgsteal) sum(pgsteal)/sum(requested)
  MGLRU off  75%                           425%
  MGLRU on   20%                           95%

  ####################################################################
  MEMCGS=128

  for ((memcg = 0; memcg < $MEMCGS; memcg++)); do
      mkdir /sys/fs/cgroup/memcg$memcg
  done

  start() {
      echo $BASHPID > /sys/fs/cgroup/memcg$memcg/cgroup.procs

      fio -name=memcg$memcg --numjobs=1 --ioengine=mmap \
          --filename=/dev/zero --size=1920M --rw=randrw \
          --rate=64m,64m --random_distribution=random \
          --fadvise_hint=0 --time_based --runtime=10h \
          --group_reporting --minimal
  }

  for ((memcg = 0; memcg < $MEMCGS; memcg++)); do
      start &
  done

  sleep 600

  for ((i = 0; i < 600; i++)); do
      echo 256m >/sys/fs/cgroup/memory.reclaim
      sleep 6
  done

  for ((memcg = 0; memcg < $MEMCGS; memcg++)); do
      grep "pgsteal " /sys/fs/cgroup/memcg$memcg/memory.stat
  done
  ####################################################################

[1]: This was obtained from running the above script (touches less
     than 256GB memory) on an EPYC 7B13 with 512GB DRAM for over an
     hour.

Yu Zhao (8):
  mm: multi-gen LRU: rename lru_gen_struct to lru_gen_folio
  mm: multi-gen LRU: rename lrugen->lists[] to lrugen->folios[]
  mm: multi-gen LRU: remove eviction fairness safeguard
  mm: multi-gen LRU: remove aging fairness safeguard
  mm: multi-gen LRU: shuffle should_run_aging()
  mm: multi-gen LRU: per-node lru_gen_folio lists
  mm: multi-gen LRU: clarify scan_control flags
  mm: multi-gen LRU: simplify arch_has_hw_pte_young() check

 Documentation/mm/multigen_lru.rst |   8 +-
 include/linux/memcontrol.h        |  10 +
 include/linux/mm_inline.h         |  25 +-
 include/linux/mmzone.h            | 127 ++++-
 mm/memcontrol.c                   |  16 +
 mm/page_alloc.c                   |   1 +
 mm/vmscan.c                       | 765 ++++++++++++++++++++----------
 mm/workingset.c                   |   4 +-
 8 files changed, 687 insertions(+), 269 deletions(-)

Comments

Yu Zhao Dec. 20, 2022, 9:49 p.m. UTC | #1 
TLDR
====
Simulated the client-server model with Chromium and Node.js pairs:
ran 1 pair per CPU and collected the number of requests (responses)
for each pair over 24 hours. (Swapped less than 10% of DRAM.)

Throughput (number of requests)       MGLRU off    MGLRU on    Change
---------------------------------------------------------------------
Total                                 9490720      16417902    +88%
Node 0                                5609841      8706039     +55%
Node 1                                3880879      7711863     +98%

Tail latency (number of requests)     MGLRU off    MGLRU on    Change
---------------------------------------------------------------------
[128s, inf)                           7            0           NaN
[64s, 128s)                           98           0           NaN
[32s, 64s)                            378          0           NaN
[16s, 32s)                            78112        416         -99%

Fairness (SD over mean requests)      MGLRU off    MGLRU on    Change
---------------------------------------------------------------------
Node 0                                12%          2%          -83%
Node 1                                9%           2%          -77%

Abbreviations
=============
CI:   confidence interval
NS:   no statistically significant difference
DUT:  device under test
ATE:  automatic test equipment

Rational
========
1. JavaScript has been mostly the most popular programming language
   for the last decade, ranking by pull requests on GitHub [1], and
   "completes its ninth year in a row as the most commonly used
   programming language" according to Stack Overflow [2].
2. ARM has the highest growth rate in the server segment. Google
   Cloud Platform [3], Microsoft Azure and Oracle offer Ampere Altra
   processors as the x86 alternative.
3. Chrome is the most used browser for web application testing,
   according to the 2021 Selenium survey [4]. Selenium is the
   standard tool for web application test automation.
4. Node.js is the standard backend JavaScript runtime environment,
   offered by all major cloud providers.

Hardware
========
DUT $ lscpu
Architecture:           aarch64
  CPU op-mode(s):       32-bit, 64-bit
  Byte Order:           Little Endian
CPU(s):                 128
  On-line CPU(s) list:  0-127
Vendor ID:              ARM
  Model name:           Neoverse-N1
    Model:              1
    Thread(s) per core: 1
    Core(s) per socket: 64
    Socket(s):          2
    Stepping:           r3p1
    Frequency boost:    disabled
    CPU max MHz:        2800.0000
    CPU min MHz:        1000.0000
    BogoMIPS:           50.00
    Flags:              fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm lrcpc dcpop asimddp ssbs
Caches (sum of all):
  L1d:                  8 MiB (128 instances)
  L1i:                  8 MiB (128 instances)
  L2:                   128 MiB (128 instances)
NUMA:
  NUMA node(s):         2
  NUMA node0 CPU(s):    0-63
  NUMA node1 CPU(s):    64-127
Vulnerabilities:
  Itlb multihit:        Not affected
  L1tf:                 Not affected
  Mds:                  Not affected
  Meltdown:             Not affected
  Mmio stale data:      Not affected
  Retbleed:             Not affected
  Spec store bypass:    Mitigation; Speculative Store Bypass disabled via prctl
  Spectre v1:           Mitigation; __user pointer sanitization
  Spectre v2:           Mitigation; CSV2, BHB
  Srbds:                Not affected
  Tsx async abort:      Not affected

DUT $ numactl -H
available: 2 nodes (0-1)
node 0 cpus: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63
node 0 size: 257730 MB
node 0 free: 256560 MB
node 1 cpus: 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127
node 1 size: 256877 MB
node 1 free: 256117 MB
node distances:
node   0   1
  0:  10  20
  1:  20  10

DUT $ cat /sys/class/nvme/nvme0/model
INTEL SSDPF21Q800GB

DUT $ cat /sys/class/nvme/nvme0/numa_node
0

DUT $ cat /sys/class/nvme/nvme1/model
INTEL SSDPF21Q800GB

DUT $ cat /sys/class/nvme/nvme1/numa_node
1

Software
========
DUT $ cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=22.04
DISTRIB_CODENAME=jammy
DISTRIB_DESCRIPTION="Ubuntu 22.04.1 LTS"

DUT $ uname -a
Linux arm 6.1.0-rc8+ #1 SMP Sat Dec 10 23:34:43 UTC 2022 aarch64 aarch64 aarch64 GNU/Linux

DUT $ cat /proc/swaps
Filename        Type         Size         Used  Priority
/dev/nvme0n1    partition    268435392    0     -2
/dev/nvme1n1    partition    268435392    0     -3

DUT $ node -v
v12.22.9

DUT $ $ chromedriver -v
ChromeDriver 105.0.5195.102 (4c16f5ffcc2da70ee2600d5db77bed423ac03a5a-refs/branch-heads/5195_55@{#4})

DUT $ python3 -c "import selenium; print(selenium.__version__)"
4.0.0a1

Procedure
=========
DUT $ cat server.js
const chunks = 8;
const size = 1024 * 1024 * 512;
const stride = 512;

const bufs = [];

for (let i = 0; i < chunks; i++) {
    bufs[i] = Buffer.alloc(size);
}

const http = require('http');

const server = http.createServer(function(req, res) {
    if (req.url != '/') {
        res.writeHead(404);
        res.end();
        return;
    }

    const rand = Math.floor(Math.random() * chunks);

    const buf = bufs[rand];
    for (let i = 0; i < buf.length; i += stride) {
        buf[i] = i;
    }

    const html = `<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<link rel="icon" href="data:,">
<title>memory test</title>
</head>
<body>
<div id="size">${buf.length}</div>
</body>
</html>`;

    res.writeHead(200, {'Content-Type': 'text/html'});
    res.end(html);
}).listen(process.argv[2]);

function exit(sig) {
    server.close(function() {
        process.exit(0);
    });
}

process.on('SIGINT', exit);

DUT $ cat client.py
import signal
import sys
import time
from selenium import webdriver

secs = [0, 1, 2, 4, 8, 16, 32, 64, 128]
hist = dict()

js = '''
const chunks = 4;
const size = 1024 * 128;
const stride = 128;

const rand = Math.floor(Math.random() * chunks);

const buf = new BigInt64Array(size * (chunks + 1));
for (let i = 0; i < buf.length; i += stride) {
    buf[i] = BigInt(i);
}

document.getElementById("size").innerHTML = "0";

return buf.length;
'''


def stop(sig, stack):
    raise KeyboardInterrupt

signal.signal(signal.SIGINT, stop)

try:
    options = webdriver.chrome.options.Options()
    options.headless = True

    driver = webdriver.Chrome(options=options)
    driver.set_script_timeout(600)
    driver.set_page_load_timeout(600)

    driver.get('http://127.0.0.1:' + sys.argv[1])

    for sec in secs:
        hist[sec] = 0

    while True:
        start = time.time()

        driver.refresh()

        size = int(driver.find_element_by_id('size').text)
        assert size > 0

        size = driver.execute_script(js)
        assert size > 0

        elapsed = time.time() - start

        for sec in reversed(secs):
            if elapsed >= sec:
                hist[sec] += 1
                break
except KeyboardInterrupt:
    print('client:', sys.argv[1], 'total: %6d,' % sum(hist.values()),
          ', '.join('%d: %6d' % (k, v) for k, v in hist.items()))

DUT $ cat js_benchmark.sh
echo 0 >/proc/sys/kernel/numa_balancing

nodes=2
memcgs=64

run() {
    trap 'wait' SIGINT

    memcg=$1
    path=/sys/fs/cgroup/memcg$memcg

    mkdir $path
    echo $BASHPID >$path/cgroup.procs

    for ((node = 0; node < $nodes; node++)); do
        port=$((nodes * memcg + node + 8000))

        numactl -N $node -m $node node server.js $port &
    done

    sleep 60

    for ((node = 0; node < $nodes; node++)); do
        port=$((nodes * memcg + node + 8000))

        numactl -N $node -m $node python3 client.py $port &
    done

    wait
}

for ((memcg = 0; memcg < $memcgs; memcg++)); do
    run $memcg &
done

sleep $((24 * 60 * 60))
trap 'wait' SIGINT
kill -INT 0

Results
=======
MGLRU off
---------
client: 8000 total:  97642, 0: 78143, 1: 17483, 2:  956, 4: 537, 8:  338, 16:  184, 32:  0, 64: 1, 128: 0
client: 8001 total:  55368, 0: 26647, 1: 25128, 2: 1618, 4: 573, 8:  544, 16:  856, 32:  1, 64: 1, 128: 0
client: 8002 total:  81680, 0: 65911, 1: 13180, 2:  778, 4: 532, 8:  498, 16:  779, 32:  2, 64: 0, 128: 0
client: 8003 total:  67382, 0: 32878, 1: 30980, 2: 2096, 4: 713, 8:  434, 16:  271, 32:  9, 64: 1, 128: 0
client: 8004 total:  72633, 0: 58276, 1: 11452, 2:  574, 4: 491, 8:  753, 16: 1086, 32:  0, 64: 1, 128: 0
client: 8005 total:  63859, 0: 32720, 1: 27363, 2: 2152, 4: 620, 8:  495, 16:  507, 32:  1, 64: 1, 128: 0
client: 8006 total:  99194, 0: 77990, 1: 19202, 2: 1008, 4: 567, 8:  329, 16:   97, 32:  1, 64: 0, 128: 0
client: 8007 total:  54722, 0: 25148, 1: 26107, 2: 1463, 4: 542, 8:  545, 16:  915, 32:  1, 64: 1, 128: 0
client: 8008 total:  92282, 0: 75384, 1: 14623, 2:  917, 4: 594, 8:  359, 16:  404, 32:  0, 64: 1, 128: 0
client: 8009 total:  59080, 0: 28775, 1: 26968, 2: 1591, 4: 567, 8:  438, 16:  732, 32:  8, 64: 1, 128: 0
client: 8010 total:  94372, 0: 74096, 1: 18259, 2:  903, 4: 520, 8:  343, 16:  250, 32:  0, 64: 1, 128: 0
client: 8011 total:  68429, 0: 32227, 1: 32569, 2: 2261, 4: 734, 8:  479, 16:  155, 32:  3, 64: 1, 128: 0
client: 8012 total:  87174, 0: 71272, 1: 13505, 2:  867, 4: 528, 8:  432, 16:  569, 32:  1, 64: 0, 128: 0
client: 8013 total:  64894, 0: 31692, 1: 29822, 2: 1837, 4: 682, 8:  435, 16:  424, 32:  1, 64: 1, 128: 0
client: 8014 total: 101275, 0: 81409, 1: 18076, 2:  949, 4: 530, 8:  268, 16:   42, 32:  0, 64: 1, 128: 0
client: 8015 total:  52166, 0: 27647, 1: 20893, 2: 1442, 4: 628, 8:  501, 16: 1040, 32: 15, 64: 0, 128: 0
client: 8016 total: 100955, 0: 79030, 1: 19997, 2: 1061, 4: 561, 8:  258, 16:   46, 32:  2, 64: 0, 128: 0
client: 8017 total:  55103, 0: 25360, 1: 26264, 2: 1514, 4: 556, 8:  483, 16:  925, 32:  0, 64: 1, 128: 0
client: 8018 total: 101058, 0: 80212, 1: 19107, 2:  923, 4: 534, 8:  255, 16:   25, 32:  1, 64: 1, 128: 0
client: 8019 total:  60505, 0: 29718, 1: 27155, 2: 1870, 4: 698, 8:  463, 16:  590, 32: 10, 64: 1, 128: 0
client: 8020 total:  88272, 0: 72067, 1: 13782, 2:  886, 4: 557, 8:  413, 16:  565, 32:  2, 64: 0, 128: 0
client: 8021 total:  61741, 0: 29676, 1: 28732, 2: 1694, 4: 591, 8:  457, 16:  587, 32:  3, 64: 1, 128: 0
client: 8022 total:  89918, 0: 73469, 1: 14042, 2:  903, 4: 555, 8:  447, 16:  501, 32:  0, 64: 1, 128: 0
client: 8023 total:  52771, 0: 26388, 1: 22833, 2: 1391, 4: 570, 8:  574, 16: 1009, 32:  5, 64: 1, 128: 0
client: 8024 total:  90558, 0: 71814, 1: 16613, 2:  852, 4: 501, 8:  320, 16:  456, 32:  1, 64: 1, 128: 0
client: 8025 total:  67534, 0: 32627, 1: 31357, 2: 2118, 4: 733, 8:  450, 16:  240, 32:  8, 64: 1, 128: 0
client: 8026 total:  89103, 0: 73391, 1: 13394, 2:  857, 4: 541, 8:  380, 16:  539, 32:  0, 64: 1, 128: 0
client: 8027 total:  64156, 0: 31064, 1: 29836, 2: 1774, 4: 610, 8:  372, 16:  497, 32:  2, 64: 0, 128: 1
client: 8028 total:  74099, 0: 59732, 1: 11598, 2:  517, 4: 456, 8:  689, 16: 1105, 32:  1, 64: 1, 128: 0
client: 8029 total:  58326, 0: 31863, 1: 22899, 2: 1652, 4: 578, 8:  468, 16:  857, 32:  9, 64: 0, 128: 0
client: 8030 total:  88146, 0: 71157, 1: 14618, 2:  881, 4: 560, 8:  418, 16:  511, 32:  0, 64: 1, 128: 0
client: 8031 total:  71225, 0: 33152, 1: 34738, 2: 2206, 4: 654, 8:  388, 16:   86, 32:  0, 64: 1, 128: 0
client: 8032 total:  86962, 0: 71089, 1: 13536, 2:  814, 4: 525, 8:  369, 16:  628, 32:  0, 64: 1, 128: 0
client: 8033 total:  65891, 0: 32062, 1: 30367, 2: 1961, 4: 709, 8:  459, 16:  321, 32: 11, 64: 1, 128: 0
client: 8034 total:  64344, 0: 53445, 1:  7714, 2:  569, 4: 445, 8:  617, 16: 1553, 32:  1, 64: 0, 128: 0
client: 8035 total:  61267, 0: 33487, 1: 24680, 2: 1397, 4: 562, 8:  443, 16:  680, 32: 18, 64: 0, 128: 0
client: 8036 total:  72258, 0: 62071, 1:  7165, 2:  659, 4: 481, 8:  488, 16: 1393, 32:  0, 64: 1, 128: 0
client: 8037 total:  49430, 0: 24902, 1: 21283, 2:  940, 4: 442, 8:  548, 16: 1314, 32:  0, 64: 1, 128: 0
client: 8038 total:  74546, 0: 60982, 1: 10722, 2:  664, 4: 484, 8:  532, 16: 1161, 32:  0, 64: 1, 128: 0
client: 8039 total:  61388, 0: 32670, 1: 25228, 2: 1696, 4: 662, 8:  499, 16:  623, 32: 10, 64: 0, 128: 0
client: 8040 total:  72193, 0: 60204, 1:  9102, 2:  633, 4: 474, 8:  567, 16: 1212, 32:  0, 64: 1, 128: 0
client: 8041 total:  70788, 0: 34450, 1: 32963, 2: 2150, 4: 681, 8:  409, 16:  133, 32:  1, 64: 0, 128: 1
client: 8042 total:  99573, 0: 79079, 1: 18589, 2: 1006, 4: 549, 8:  291, 16:   57, 32:  2, 64: 0, 128: 0
client: 8043 total:  61150, 0: 30443, 1: 27235, 2: 1833, 4: 586, 8:  451, 16:  589, 32: 13, 64: 0, 128: 0
client: 8044 total:  92953, 0: 73432, 1: 17453, 2:  858, 4: 529, 8:  320, 16:  360, 32:  0, 64: 1, 128: 0
client: 8045 total:  67159, 0: 31960, 1: 31586, 2: 2243, 4: 657, 8:  444, 16:  266, 32:  2, 64: 1, 128: 0
client: 8046 total:  89958, 0: 72951, 1: 14700, 2:  917, 4: 528, 8:  371, 16:  490, 32:  1, 64: 0, 128: 0
client: 8047 total:  64967, 0: 30897, 1: 30717, 2: 1828, 4: 638, 8:  499, 16:  386, 32:  1, 64: 1, 128: 0
client: 8048 total:  74985, 0: 62435, 1:  9756, 2:  697, 4: 447, 8:  483, 16: 1167, 32:  0, 64: 0, 128: 0
client: 8049 total:  68061, 0: 34043, 1: 30689, 2: 1923, 4: 704, 8:  427, 16:  268, 32:  6, 64: 1, 128: 0
client: 8050 total:  92109, 0: 74836, 1: 15038, 2:  887, 4: 561, 8:  395, 16:  391, 32:  0, 64: 1, 128: 0
client: 8051 total:  61302, 0: 29354, 1: 28320, 2: 1873, 4: 678, 8:  541, 16:  530, 32:  5, 64: 1, 128: 0
client: 8052 total:  93586, 0: 75682, 1: 15680, 2:  954, 4: 606, 8:  353, 16:  309, 32:  2, 64: 0, 128: 0
client: 8053 total:  62111, 0: 29798, 1: 28888, 2: 1761, 4: 649, 8:  478, 16:  534, 32:  2, 64: 1, 128: 0
client: 8054 total:  77598, 0: 63967, 1: 10939, 2:  565, 4: 449, 8:  656, 16: 1021, 32:  0, 64: 1, 128: 0
client: 8055 total:  52696, 0: 29572, 1: 19464, 2: 1470, 4: 573, 8:  496, 16: 1108, 32: 12, 64: 1, 128: 0
client: 8056 total: 101806, 0: 79977, 1: 19972, 2: 1061, 4: 515, 8:  245, 16:   35, 32:  0, 64: 1, 128: 0
client: 8057 total:  51714, 0: 23892, 1: 24298, 2: 1316, 4: 577, 8:  556, 16: 1074, 32:  0, 64: 1, 128: 0
client: 8058 total:  99891, 0: 78935, 1: 19039, 2: 1029, 4: 524, 8:  308, 16:   55, 32:  0, 64: 1, 128: 0
client: 8059 total:  61197, 0: 29219, 1: 28484, 2: 1797, 4: 639, 8:  476, 16:  580, 32:  1, 64: 1, 128: 0
client: 8060 total:  84812, 0: 67853, 1: 14444, 2:  822, 4: 472, 8:  457, 16:  763, 32:  0, 64: 1, 128: 0
client: 8061 total:  55538, 0: 25740, 1: 26231, 2: 1553, 4: 570, 8:  570, 16:  873, 32:  0, 64: 1, 128: 0
client: 8062 total: 100386, 0: 78818, 1: 19634, 2:  971, 4: 580, 8:  322, 16:   60, 32:  1, 64: 0, 128: 0
client: 8063 total:  54708, 0: 24943, 1: 26354, 2: 1391, 4: 561, 8:  547, 16:  909, 32:  2, 64: 1, 128: 0
client: 8064 total:  79777, 0: 66977, 1: 10038, 2:  776, 4: 535, 8:  464, 16:  986, 32:  0, 64: 1, 128: 0
client: 8065 total:  58150, 0: 27960, 1: 27001, 2: 1356, 4: 512, 8:  510, 16:  810, 32:  0, 64: 1, 128: 0
client: 8066 total:  99609, 0: 79518, 1: 18238, 2:  965, 4: 535, 8:  281, 16:   71, 32:  0, 64: 1, 128: 0
client: 8067 total:  55633, 0: 28332, 1: 23697, 2: 1602, 4: 607, 8:  546, 16:  839, 32:  9, 64: 1, 128: 0
client: 8068 total:  92506, 0: 73032, 1: 17323, 2:  910, 4: 575, 8:  347, 16:  318, 32:  0, 64: 1, 128: 0
client: 8069 total:  66166, 0: 31316, 1: 31148, 2: 2232, 4: 737, 8:  442, 16:  287, 32:  3, 64: 1, 128: 0
client: 8070 total:  73162, 0: 59514, 1: 10795, 2:  703, 4: 498, 8:  472, 16: 1179, 32:  0, 64: 1, 128: 0
client: 8071 total:  72629, 0: 35681, 1: 33578, 2: 2276, 4: 650, 8:  373, 16:   69, 32:  1, 64: 1, 128: 0
client: 8072 total:  89214, 0: 72864, 1: 14051, 2:  860, 4: 522, 8:  339, 16:  577, 32:  0, 64: 1, 128: 0
client: 8073 total:  62734, 0: 30110, 1: 29280, 2: 1779, 4: 590, 8:  445, 16:  529, 32:  0, 64: 1, 128: 0
client: 8074 total:  83958, 0: 67661, 1: 13755, 2:  820, 4: 537, 8:  459, 16:  725, 32:  0, 64: 1, 128: 0
client: 8075 total:  58355, 0: 29682, 1: 25160, 2: 1641, 4: 590, 8:  520, 16:  757, 32:  4, 64: 1, 128: 0
client: 8076 total:  77144, 0: 62866, 1: 11623, 2:  575, 4: 463, 8:  655, 16:  961, 32:  0, 64: 1, 128: 0
client: 8077 total:  60316, 0: 32464, 1: 24378, 2: 1656, 4: 650, 8:  453, 16:  699, 32: 15, 64: 1, 128: 0
client: 8078 total: 100348, 0: 79106, 1: 19344, 2: 1042, 4: 551, 8:  264, 16:   40, 32:  0, 64: 1, 128: 0
client: 8079 total:  61621, 0: 29765, 1: 28253, 2: 1938, 4: 638, 8:  464, 16:  561, 32:  1, 64: 1, 128: 0
client: 8080 total:  77042, 0: 62427, 1: 11994, 2:  560, 4: 440, 8:  620, 16: 1000, 32:  0, 64: 1, 128: 0
client: 8081 total:  64302, 0: 33050, 1: 27531, 2: 2110, 4: 680, 8:  437, 16:  490, 32:  3, 64: 1, 128: 0
client: 8082 total:  68569, 0: 56382, 1:  9139, 2:  581, 4: 454, 8:  603, 16: 1409, 32:  0, 64: 1, 128: 0
client: 8083 total:  60421, 0: 33506, 1: 23569, 2: 1533, 4: 605, 8:  462, 16:  734, 32: 11, 64: 1, 128: 0
client: 8084 total:  90413, 0: 74210, 1: 13899, 2:  878, 4: 534, 8:  382, 16:  508, 32:  2, 64: 0, 128: 0
client: 8085 total:  61547, 0: 29591, 1: 28607, 2: 1693, 4: 613, 8:  454, 16:  586, 32:  2, 64: 0, 128: 1
client: 8086 total:  73575, 0: 59394, 1: 11463, 2:  521, 4: 461, 8:  639, 16: 1096, 32:  0, 64: 1, 128: 0
client: 8087 total:  66359, 0: 34090, 1: 28727, 2: 2050, 4: 662, 8:  439, 16:  379, 32: 11, 64: 1, 128: 0
client: 8088 total: 100232, 0: 79400, 1: 18999, 2: 1007, 4: 541, 8:  242, 16:   42, 32:  0, 64: 1, 128: 0
client: 8089 total:  60029, 0: 29965, 1: 26429, 2: 1793, 4: 702, 8:  531, 16:  600, 32:  8, 64: 1, 128: 0
client: 8090 total:  83306, 0: 67111, 1: 13867, 2:  588, 4: 436, 8:  596, 16:  707, 32:  0, 64: 1, 128: 0
client: 8091 total:  63605, 0: 32317, 1: 27707, 2: 1981, 4: 613, 8:  488, 16:  490, 32:  8, 64: 1, 128: 0
client: 8092 total:  77397, 0: 63243, 1: 11578, 2:  512, 4: 446, 8:  595, 16: 1022, 32:  0, 64: 1, 128: 0
client: 8093 total:  58603, 0: 31124, 1: 23833, 2: 1712, 4: 650, 8:  539, 16:  734, 32: 10, 64: 0, 128: 1
client: 8094 total:  90750, 0: 72661, 1: 15890, 2:  861, 4: 514, 8:  377, 16:  445, 32:  2, 64: 0, 128: 0
client: 8095 total:  60656, 0: 31003, 1: 26152, 2: 1722, 4: 660, 8:  483, 16:  627, 32:  8, 64: 0, 128: 1
client: 8096 total:  95700, 0: 76523, 1: 17194, 2:  867, 4: 505, 8:  357, 16:  253, 32:  0, 64: 1, 128: 0
client: 8097 total:  55921, 0: 27431, 1: 24725, 2: 1731, 4: 642, 8:  575, 16:  815, 32:  1, 64: 1, 128: 0
client: 8098 total:  86615, 0: 71408, 1: 12726, 2:  872, 4: 497, 8:  392, 16:  719, 32:  1, 64: 0, 128: 0
client: 8099 total:  54340, 0: 25749, 1: 25300, 2: 1258, 4: 546, 8:  471, 16: 1012, 32:  3, 64: 0, 128: 1
client: 8100 total: 101811, 0: 81236, 1: 18803, 2:  992, 4: 517, 8:  230, 16:   32, 32:  0, 64: 1, 128: 0
client: 8101 total:  55106, 0: 28254, 1: 23328, 2: 1534, 4: 607, 8:  477, 16:  886, 32: 19, 64: 1, 128: 0
client: 8102 total:  97187, 0: 78106, 1: 16953, 2:  999, 4: 531, 8:  329, 16:  268, 32:  0, 64: 1, 128: 0
client: 8103 total:  49235, 0: 22971, 1: 22932, 2: 1126, 4: 447, 8:  508, 16: 1250, 32:  0, 64: 1, 128: 0
client: 8104 total:  79003, 0: 64417, 1: 11910, 2:  802, 4: 537, 8:  418, 16:  918, 32:  1, 64: 0, 128: 0
client: 8105 total:  70947, 0: 33860, 1: 33646, 2: 2262, 4: 687, 8:  387, 16:  104, 32:  0, 64: 1, 128: 0
client: 8106 total:  95571, 0: 75702, 1: 17737, 2:  965, 4: 526, 8:  314, 16:  326, 32:  1, 64: 0, 128: 0
client: 8107 total:  53974, 0: 25299, 1: 25117, 2: 1485, 4: 566, 8:  540, 16:  965, 32:  2, 64: 0, 128: 0
client: 8108 total: 100811, 0: 79960, 1: 19072, 2:  952, 4: 532, 8:  259, 16:   35, 32:  0, 64: 1, 128: 0
client: 8109 total:  57953, 0: 29463, 1: 24794, 2: 1693, 4: 694, 8:  626, 16:  671, 32: 11, 64: 1, 128: 0
client: 8110 total:  99779, 0: 79272, 1: 18659, 2:  989, 4: 503, 8:  286, 16:   69, 32:  0, 64: 1, 128: 0
client: 8111 total:  58895, 0: 29628, 1: 25699, 2: 1750, 4: 641, 8:  489, 16:  677, 32: 10, 64: 1, 128: 0
client: 8112 total: 100120, 0: 78217, 1: 20003, 2: 1002, 4: 564, 8:  291, 16:   42, 32:  1, 64: 0, 128: 0
client: 8113 total:  60226, 0: 27861, 1: 28981, 2: 1684, 4: 588, 8:  455, 16:  655, 32:  1, 64: 0, 128: 1
client: 8114 total:  76007, 0: 62207, 1: 10978, 2:  736, 4: 553, 8:  506, 16: 1026, 32:  0, 64: 1, 128: 0
client: 8115 total:  68739, 0: 33453, 1: 31888, 2: 2083, 4: 668, 8:  409, 16:  234, 32:  3, 64: 1, 128: 0
client: 8116 total:  72167, 0: 60625, 1:  8528, 2:  751, 4: 498, 8:  536, 16: 1228, 32:  0, 64: 1, 128: 0
client: 8117 total:  63070, 0: 32596, 1: 27312, 2: 1575, 4: 580, 8:  418, 16:  573, 32: 15, 64: 1, 128: 0
client: 8118 total:  94387, 0: 76269, 1: 15914, 2:  997, 4: 566, 8:  346, 16:  294, 32:  0, 64: 1, 128: 0
client: 8119 total:  60738, 0: 29308, 1: 28053, 2: 1700, 4: 610, 8:  441, 16:  623, 32:  2, 64: 1, 128: 0
client: 8120 total:  71831, 0: 58997, 1:  9899, 2:  588, 4: 506, 8:  645, 16: 1195, 32:  0, 64: 1, 128: 0
client: 8121 total:  61332, 0: 32584, 1: 25465, 2: 1524, 4: 619, 8:  464, 16:  662, 32: 13, 64: 1, 128: 0
client: 8122 total:  90574, 0: 72577, 1: 15600, 2:  958, 4: 540, 8:  411, 16:  487, 32:  0, 64: 1, 128: 0
client: 8123 total:  55340, 0: 26122, 1: 25865, 2: 1345, 4: 557, 8:  499, 16:  950, 32:  1, 64: 1, 128: 0
client: 8124 total:  65043, 0: 53995, 1:  7914, 2:  437, 4: 399, 8:  767, 16: 1530, 32:  0, 64: 1, 128: 0
client: 8125 total:  58368, 0: 33484, 1: 21438, 2: 1504, 4: 555, 8:  516, 16:  859, 32: 11, 64: 1, 128: 0
client: 8126 total:  99912, 0: 80003, 1: 18056, 2:  973, 4: 511, 8:  279, 16:   89, 32:  0, 64: 1, 128: 0
client: 8127 total:  58941, 0: 28988, 1: 26433, 2: 1744, 4: 598, 8:  476, 16:  695, 32:  6, 64: 1, 128: 0

MGLRU on
--------
client: 8000 total: 135911, 0: 131546, 1:  943, 2:  801, 4: 642, 8: 1977, 16:    2, 32:  0, 64: 0, 128: 0
client: 8001 total: 120756, 0: 116009, 1: 1341, 2:  854, 4: 675, 8: 1876, 16:    1, 32:  0, 64: 0, 128: 0
client: 8002 total: 140312, 0: 136194, 1:  930, 2:  766, 4: 701, 8: 1716, 16:    5, 32:  0, 64: 0, 128: 0
client: 8003 total: 120951, 0: 116166, 1: 1432, 2:  895, 4: 648, 8: 1807, 16:    3, 32:  0, 64: 0, 128: 0
client: 8004 total: 134239, 0: 129877, 1:  883, 2:  774, 4: 690, 8: 2011, 16:    4, 32:  0, 64: 0, 128: 0
client: 8005 total: 119151, 0: 114323, 1: 1373, 2:  880, 4: 606, 8: 1966, 16:    3, 32:  0, 64: 0, 128: 0
client: 8006 total: 140413, 0: 136236, 1:  945, 2:  806, 4: 688, 8: 1737, 16:    1, 32:  0, 64: 0, 128: 0
client: 8007 total: 120694, 0: 115888, 1: 1511, 2:  871, 4: 599, 8: 1822, 16:    3, 32:  0, 64: 0, 128: 0
client: 8008 total: 135646, 0: 131378, 1:  879, 2:  742, 4: 669, 8: 1973, 16:    5, 32:  0, 64: 0, 128: 0
client: 8009 total: 118450, 0: 113578, 1: 1303, 2:  952, 4: 610, 8: 2003, 16:    4, 32:  0, 64: 0, 128: 0
client: 8010 total: 134853, 0: 130437, 1:  951, 2:  806, 4: 664, 8: 1989, 16:    6, 32:  0, 64: 0, 128: 0
client: 8011 total: 121233, 0: 116564, 1: 1292, 2:  866, 4: 608, 8: 1902, 16:    1, 32:  0, 64: 0, 128: 0
client: 8012 total: 130700, 0: 126167, 1:  899, 2:  771, 4: 745, 8: 2112, 16:    6, 32:  0, 64: 0, 128: 0
client: 8013 total: 122335, 0: 117575, 1: 1407, 2:  931, 4: 603, 8: 1817, 16:    2, 32:  0, 64: 0, 128: 0
client: 8014 total: 132725, 0: 128209, 1: 1012, 2:  777, 4: 710, 8: 2010, 16:    7, 32:  0, 64: 0, 128: 0
client: 8015 total: 126804, 0: 122228, 1: 1484, 2:  964, 4: 602, 8: 1525, 16:    1, 32:  0, 64: 0, 128: 0
client: 8016 total: 137896, 0: 133606, 1:  944, 2:  774, 4: 728, 8: 1840, 16:    4, 32:  0, 64: 0, 128: 0
client: 8017 total: 119403, 0: 114641, 1: 1330, 2:  929, 4: 578, 8: 1924, 16:    1, 32:  0, 64: 0, 128: 0
client: 8018 total: 134848, 0: 130415, 1:  972, 2:  742, 4: 710, 8: 2007, 16:    2, 32:  0, 64: 0, 128: 0
client: 8019 total: 119305, 0: 114515, 1: 1370, 2:  895, 4: 577, 8: 1946, 16:    2, 32:  0, 64: 0, 128: 0
client: 8020 total: 140862, 0: 136704, 1:  932, 2:  805, 4: 695, 8: 1723, 16:    3, 32:  0, 64: 0, 128: 0
client: 8021 total: 120837, 0: 116074, 1: 1410, 2:  901, 4: 617, 8: 1831, 16:    4, 32:  0, 64: 0, 128: 0
client: 8022 total: 132710, 0: 128278, 1:  902, 2:  783, 4: 664, 8: 2079, 16:    4, 32:  0, 64: 0, 128: 0
client: 8023 total: 126671, 0: 122176, 1: 1420, 2:  916, 4: 576, 8: 1578, 16:    5, 32:  0, 64: 0, 128: 0
client: 8024 total: 135185, 0: 130845, 1:  878, 2:  815, 4: 669, 8: 1977, 16:    1, 32:  0, 64: 0, 128: 0
client: 8025 total: 122042, 0: 117414, 1: 1321, 2:  911, 4: 579, 8: 1814, 16:    3, 32:  0, 64: 0, 128: 0
client: 8026 total: 135082, 0: 130656, 1:  920, 2:  778, 4: 752, 8: 1974, 16:    2, 32:  0, 64: 0, 128: 0
client: 8027 total: 123020, 0: 118290, 1: 1440, 2:  944, 4: 583, 8: 1757, 16:    6, 32:  0, 64: 0, 128: 0
client: 8028 total: 135350, 0: 130949, 1:  880, 2:  846, 4: 705, 8: 1967, 16:    3, 32:  0, 64: 0, 128: 0
client: 8029 total: 119721, 0: 114996, 1: 1299, 2:  920, 4: 621, 8: 1882, 16:    3, 32:  0, 64: 0, 128: 0
client: 8030 total: 131991, 0: 127411, 1:  934, 2:  826, 4: 673, 8: 2144, 16:    3, 32:  0, 64: 0, 128: 0
client: 8031 total: 123002, 0: 118304, 1: 1385, 2:  939, 4: 652, 8: 1722, 16:    0, 32:  0, 64: 0, 128: 0
client: 8032 total: 130133, 0: 125547, 1:  952, 2:  758, 4: 656, 8: 2213, 16:    7, 32:  0, 64: 0, 128: 0
client: 8033 total: 125114, 0: 120632, 1: 1307, 2:  867, 4: 666, 8: 1640, 16:    2, 32:  0, 64: 0, 128: 0
client: 8034 total: 137708, 0: 133396, 1:  977, 2:  776, 4: 699, 8: 1853, 16:    7, 32:  0, 64: 0, 128: 0
client: 8035 total: 122820, 0: 118162, 1: 1377, 2:  899, 4: 629, 8: 1753, 16:    0, 32:  0, 64: 0, 128: 0
client: 8036 total: 132909, 0: 128405, 1:  935, 2:  791, 4: 664, 8: 2106, 16:    8, 32:  0, 64: 0, 128: 0
client: 8037 total: 121966, 0: 117335, 1: 1295, 2:  927, 4: 589, 8: 1816, 16:    4, 32:  0, 64: 0, 128: 0
client: 8038 total: 137967, 0: 133673, 1:  961, 2:  802, 4: 680, 8: 1847, 16:    4, 32:  0, 64: 0, 128: 0
client: 8039 total: 122542, 0: 117875, 1: 1410, 2:  890, 4: 598, 8: 1764, 16:    5, 32:  0, 64: 0, 128: 0
client: 8040 total: 137028, 0: 132749, 1:  953, 2:  774, 4: 649, 8: 1901, 16:    2, 32:  0, 64: 0, 128: 0
client: 8041 total: 122337, 0: 117649, 1: 1334, 2:  942, 4: 621, 8: 1787, 16:    4, 32:  0, 64: 0, 128: 0
client: 8042 total: 133651, 0: 129143, 1: 1012, 2:  786, 4: 680, 8: 2024, 16:    6, 32:  0, 64: 0, 128: 0
client: 8043 total: 124615, 0: 120053, 1: 1348, 2:  948, 4: 595, 8: 1667, 16:    4, 32:  0, 64: 0, 128: 0
client: 8044 total: 135503, 0: 131208, 1:  907, 2:  748, 4: 651, 8: 1982, 16:    7, 32:  0, 64: 0, 128: 0
client: 8045 total: 120529, 0: 115721, 1: 1400, 2:  943, 4: 601, 8: 1860, 16:    4, 32:  0, 64: 0, 128: 0
client: 8046 total: 131720, 0: 127244, 1:  865, 2:  768, 4: 754, 8: 2086, 16:    3, 32:  0, 64: 0, 128: 0
client: 8047 total: 124835, 0: 120237, 1: 1351, 2:  912, 4: 667, 8: 1666, 16:    2, 32:  0, 64: 0, 128: 0
client: 8048 total: 135540, 0: 131116, 1:  991, 2:  772, 4: 737, 8: 1917, 16:    7, 32:  0, 64: 0, 128: 0
client: 8049 total: 118887, 0: 114049, 1: 1357, 2:  900, 4: 649, 8: 1929, 16:    3, 32:  0, 64: 0, 128: 0
client: 8050 total: 131503, 0: 126909, 1:  951, 2:  809, 4: 706, 8: 2126, 16:    2, 32:  0, 64: 0, 128: 0
client: 8051 total: 119694, 0: 114937, 1: 1317, 2:  862, 4: 657, 8: 1920, 16:    1, 32:  0, 64: 0, 128: 0
client: 8052 total: 133973, 0: 129595, 1:  845, 2:  723, 4: 751, 8: 2059, 16:    0, 32:  0, 64: 0, 128: 0
client: 8053 total: 121011, 0: 116242, 1: 1372, 2:  930, 4: 610, 8: 1857, 16:    0, 32:  0, 64: 0, 128: 0
client: 8054 total: 139674, 0: 135515, 1:  969, 2:  774, 4: 663, 8: 1750, 16:    3, 32:  0, 64: 0, 128: 0
client: 8055 total: 121594, 0: 116707, 1: 1508, 2:  944, 4: 627, 8: 1808, 16:    0, 32:  0, 64: 0, 128: 0
client: 8056 total: 139949, 0: 135796, 1:  935, 2:  766, 4: 719, 8: 1727, 16:    6, 32:  0, 64: 0, 128: 0
client: 8057 total: 118338, 0: 113433, 1: 1398, 2:  900, 4: 664, 8: 1938, 16:    5, 32:  0, 64: 0, 128: 0
client: 8058 total: 133214, 0: 128824, 1:  833, 2:  828, 4: 667, 8: 2060, 16:    2, 32:  0, 64: 0, 128: 0
client: 8059 total: 126622, 0: 122081, 1: 1401, 2:  964, 4: 598, 8: 1576, 16:    2, 32:  0, 64: 0, 128: 0
client: 8060 total: 135070, 0: 130693, 1:  973, 2:  746, 4: 707, 8: 1949, 16:    2, 32:  0, 64: 0, 128: 0
client: 8061 total: 117996, 0: 113126, 1: 1263, 2:  922, 4: 659, 8: 2024, 16:    2, 32:  0, 64: 0, 128: 0
client: 8062 total: 141314, 0: 137223, 1:  921, 2:  752, 4: 732, 8: 1684, 16:    2, 32:  0, 64: 0, 128: 0
client: 8063 total: 115228, 0: 110205, 1: 1352, 2:  927, 4: 623, 8: 2118, 16:    3, 32:  0, 64: 0, 128: 0
client: 8064 total: 142955, 0: 138947, 1:  956, 2:  769, 4: 691, 8: 1590, 16:    2, 32:  0, 64: 0, 128: 0
client: 8065 total: 122755, 0: 118086, 1: 1407, 2:  908, 4: 651, 8: 1701, 16:    2, 32:  0, 64: 0, 128: 0
client: 8066 total: 136624, 0: 132286, 1:  957, 2:  767, 4: 704, 8: 1906, 16:    4, 32:  0, 64: 0, 128: 0
client: 8067 total: 120349, 0: 115590, 1: 1421, 2:  894, 4: 542, 8: 1902, 16:    0, 32:  0, 64: 0, 128: 0
client: 8068 total: 131028, 0: 126382, 1:  921, 2:  853, 4: 677, 8: 2189, 16:    6, 32:  0, 64: 0, 128: 0
client: 8069 total: 117941, 0: 113116, 1: 1297, 2:  870, 4: 607, 8: 2050, 16:    1, 32:  0, 64: 0, 128: 0
client: 8070 total: 137469, 0: 133203, 1:  864, 2:  786, 4: 692, 8: 1921, 16:    3, 32:  0, 64: 0, 128: 0
client: 8071 total: 116240, 0: 111288, 1: 1371, 2:  872, 4: 612, 8: 2094, 16:    3, 32:  0, 64: 0, 128: 0
client: 8072 total: 137278, 0: 132999, 1:  926, 2:  782, 4: 762, 8: 1807, 16:    2, 32:  0, 64: 0, 128: 0
client: 8073 total: 121421, 0: 116615, 1: 1469, 2:  942, 4: 566, 8: 1827, 16:    2, 32:  0, 64: 0, 128: 0
client: 8074 total: 130281, 0: 125717, 1:  906, 2:  789, 4: 632, 8: 2228, 16:    9, 32:  0, 64: 0, 128: 0
client: 8075 total: 120662, 0: 115944, 1: 1307, 2:  919, 4: 573, 8: 1918, 16:    1, 32:  0, 64: 0, 128: 0
client: 8076 total: 139454, 0: 135179, 1:  969, 2:  842, 4: 692, 8: 1769, 16:    3, 32:  0, 64: 0, 128: 0
client: 8077 total: 120573, 0: 115764, 1: 1440, 2:  917, 4: 615, 8: 1835, 16:    2, 32:  0, 64: 0, 128: 0
client: 8078 total: 137138, 0: 132847, 1:  912, 2:  770, 4: 703, 8: 1903, 16:    3, 32:  0, 64: 0, 128: 0
client: 8079 total: 116102, 0: 111135, 1: 1343, 2:  908, 4: 607, 8: 2108, 16:    1, 32:  0, 64: 0, 128: 0
client: 8080 total: 137181, 0: 132934, 1:  880, 2:  758, 4: 711, 8: 1896, 16:    2, 32:  0, 64: 0, 128: 0
client: 8081 total: 120698, 0: 115918, 1: 1451, 2:  880, 4: 589, 8: 1856, 16:    4, 32:  0, 64: 0, 128: 0
client: 8082 total: 140708, 0: 136601, 1:  963, 2:  743, 4: 679, 8: 1718, 16:    4, 32:  0, 64: 0, 128: 0
client: 8083 total: 120397, 0: 115540, 1: 1465, 2:  968, 4: 554, 8: 1867, 16:    3, 32:  0, 64: 0, 128: 0
client: 8084 total: 136258, 0: 131848, 1:  946, 2:  853, 4: 710, 8: 1899, 16:    2, 32:  0, 64: 0, 128: 0
client: 8085 total: 122964, 0: 118257, 1: 1470, 2:  906, 4: 598, 8: 1729, 16:    4, 32:  0, 64: 0, 128: 0
client: 8086 total: 139809, 0: 135609, 1:  967, 2:  769, 4: 670, 8: 1792, 16:    2, 32:  0, 64: 0, 128: 0
client: 8087 total: 112384, 0: 107326, 1: 1285, 2:  855, 4: 659, 8: 2254, 16:    5, 32:  0, 64: 0, 128: 0
client: 8088 total: 139801, 0: 135647, 1:  904, 2:  770, 4: 707, 8: 1769, 16:    4, 32:  0, 64: 0, 128: 0
client: 8089 total: 116987, 0: 112011, 1: 1402, 2:  882, 4: 657, 8: 2032, 16:    3, 32:  0, 64: 0, 128: 0
client: 8090 total: 136223, 0: 131911, 1:  950, 2:  771, 4: 649, 8: 1938, 16:    4, 32:  0, 64: 0, 128: 0
client: 8091 total: 119708, 0: 114922, 1: 1342, 2:  887, 4: 629, 8: 1924, 16:    4, 32:  0, 64: 0, 128: 0
client: 8092 total: 135010, 0: 130541, 1: 1016, 2:  773, 4: 673, 8: 2005, 16:    2, 32:  0, 64: 0, 128: 0
client: 8093 total: 123205, 0: 118580, 1: 1321, 2:  894, 4: 734, 8: 1670, 16:    6, 32:  0, 64: 0, 128: 0
client: 8094 total: 136616, 0: 132297, 1:  935, 2:  768, 4: 640, 8: 1968, 16:    8, 32:  0, 64: 0, 128: 0
client: 8095 total: 117438, 0: 112540, 1: 1405, 2:  856, 4: 600, 8: 2034, 16:    3, 32:  0, 64: 0, 128: 0
client: 8096 total: 134197, 0: 129768, 1:  918, 2:  797, 4: 700, 8: 2013, 16:    1, 32:  0, 64: 0, 128: 0
client: 8097 total: 120484, 0: 115747, 1: 1342, 2:  895, 4: 588, 8: 1908, 16:    4, 32:  0, 64: 0, 128: 0
client: 8098 total: 140243, 0: 136068, 1:  913, 2:  808, 4: 722, 8: 1726, 16:    6, 32:  0, 64: 0, 128: 0
client: 8099 total: 120246, 0: 115400, 1: 1421, 2:  932, 4: 623, 8: 1870, 16:    0, 32:  0, 64: 0, 128: 0
client: 8100 total: 137723, 0: 133391, 1: 1009, 2:  808, 4: 650, 8: 1863, 16:    2, 32:  0, 64: 0, 128: 0
client: 8101 total: 120033, 0: 115244, 1: 1390, 2:  879, 4: 660, 8: 1857, 16:    3, 32:  0, 64: 0, 128: 0
client: 8102 total: 133661, 0: 129007, 1: 1059, 2:  851, 4: 691, 8: 2048, 16:    5, 32:  0, 64: 0, 128: 0
client: 8103 total: 121359, 0: 116670, 1: 1393, 2:  850, 4: 592, 8: 1848, 16:    6, 32:  0, 64: 0, 128: 0
client: 8104 total: 140702, 0: 136607, 1:  918, 2:  758, 4: 675, 8: 1735, 16:    9, 32:  0, 64: 0, 128: 0
client: 8105 total: 117126, 0: 112234, 1: 1310, 2:  935, 4: 658, 8: 1989, 16:    0, 32:  0, 64: 0, 128: 0
client: 8106 total: 137042, 0: 132779, 1:  882, 2:  816, 4: 707, 8: 1856, 16:    2, 32:  0, 64: 0, 128: 0
client: 8107 total: 121781, 0: 116961, 1: 1487, 2:  920, 4: 614, 8: 1795, 16:    4, 32:  0, 64: 0, 128: 0
client: 8108 total: 131094, 0: 126590, 1:  861, 2:  800, 4: 657, 8: 2182, 16:    4, 32:  0, 64: 0, 128: 0
client: 8109 total: 121888, 0: 117200, 1: 1333, 2:  952, 4: 620, 8: 1781, 16:    2, 32:  0, 64: 0, 128: 0
client: 8110 total: 136023, 0: 131748, 1:  895, 2:  751, 4: 739, 8: 1885, 16:    5, 32:  0, 64: 0, 128: 0
client: 8111 total: 112796, 0: 107705, 1: 1278, 2:  907, 4: 634, 8: 2269, 16:    3, 32:  0, 64: 0, 128: 0
client: 8112 total: 132701, 0: 128249, 1:  840, 2:  806, 4: 696, 8: 2106, 16:    4, 32:  0, 64: 0, 128: 0
client: 8113 total: 121891, 0: 117107, 1: 1438, 2:  915, 4: 621, 8: 1805, 16:    5, 32:  0, 64: 0, 128: 0
client: 8114 total: 139379, 0: 135237, 1:  879, 2:  777, 4: 655, 8: 1828, 16:    3, 32:  0, 64: 0, 128: 0
client: 8115 total: 114118, 0: 108925, 1: 1390, 2:  939, 4: 709, 8: 2152, 16:    3, 32:  0, 64: 0, 128: 0
client: 8116 total: 129121, 0: 124433, 1:  992, 2:  803, 4: 620, 8: 2269, 16:    4, 32:  0, 64: 0, 128: 0
client: 8117 total: 123295, 0: 118644, 1: 1339, 2:  927, 4: 650, 8: 1731, 16:    4, 32:  0, 64: 0, 128: 0
client: 8118 total: 139996, 0: 135890, 1:  880, 2:  797, 4: 652, 8: 1773, 16:    4, 32:  0, 64: 0, 128: 0
client: 8119 total: 118785, 0: 114003, 1: 1371, 2:  866, 4: 577, 8: 1968, 16:    0, 32:  0, 64: 0, 128: 0
client: 8120 total: 139403, 0: 135284, 1:  848, 2:  798, 4: 700, 8: 1769, 16:    4, 32:  0, 64: 0, 128: 0
client: 8121 total: 118842, 0: 114098, 1: 1286, 2:  849, 4: 631, 8: 1974, 16:    4, 32:  0, 64: 0, 128: 0
client: 8122 total: 139147, 0: 134951, 1:  931, 2:  804, 4: 669, 8: 1792, 16:    0, 32:  0, 64: 0, 128: 0
client: 8123 total: 117281, 0: 112432, 1: 1304, 2:  918, 4: 643, 8: 1981, 16:    3, 32:  0, 64: 0, 128: 0
client: 8124 total: 132181, 0: 127728, 1:  847, 2:  805, 4: 683, 8: 2112, 16:    6, 32:  0, 64: 0, 128: 0
client: 8125 total: 123172, 0: 118464, 1: 1397, 2:  928, 4: 645, 8: 1737, 16:    1, 32:  0, 64: 0, 128: 0
client: 8126 total: 134014, 0: 129499, 1:  939, 2:  834, 4: 692, 8: 2049, 16:    1, 32:  0, 64: 0, 128: 0
client: 8127 total: 120439, 0: 115674, 1: 1342, 2:  869, 4: 666, 8: 1887, 16:    1, 32:  0, 64: 0, 128: 0

References
==========
[1] https://madnight.github.io/githut/#/pull_requests/2022/1
[2] https://insights.stackoverflow.com/survey/2021#technology
[3] https://cloud.google.com/blog/products/compute/tau-t2a-is-first-compute-engine-vm-on-an-arm-chip
[4] https://www.selenium.dev/blog/2021/selenium-survey-results/