fio 테스트

개발사에서 2TB 이상의 DB disk size 를 요구 받았습니다.

AWS 로는 가능하지만 KT 는 일반적인 디스크 방식으로는 가능하지 않으며, 이를 구현하기 위해서는 NAS(NFS,CIFS,ISCSI 지원) 와 zadara 를 사용해야 합니다.

zadara는 복잡도 증가로 패스하였고, NAS 를 iscsi 방식으로 연결하였습니다. (NFS는 로컬 디바이스로 인식되지 않아 사용율 모니터링이 되지 않음)

iscsi로 연결하고 iops 확인을 위해 fio 를 이용하여 테스트 한 내용입니다.

대역폭 보장이 안된다는 단점이 있어 불안하긴 합니다. 어쨌든 테스트 기록을 위해 남겨놓습니다.

Write TEST (fio_test_file을 1GB로 16번 생성하여 리포트)

fio --directory=/data --name fio_test_file --direct=1 --rw=randwrite --bs=16k --size=1G --numjobs=16 --time_based --runtime=180 --group_reporting --norandommap 

Jobs: 16 (f=16): [w(16)] [100.0% done] [0KB/39928KB/0KB /s] [0/9982/0 iops] [eta 00m:00s]

fio_test_file: (groupid=0, jobs=16): err= 0: pid=29267: Tue Jan  5 11:20:53 2016

  write: io=7069.9MB, bw=40219KB/s, iops=10054, runt=180001msec

    clat (usec): min=272, max=788685, avg=1587.10, stdev=7423.31

     lat (usec): min=272, max=788685, avg=1587.43, stdev=7423.31

    clat percentiles (usec):

     |  1.00th=[  466],  5.00th=[  580], 10.00th=[  660], 20.00th=[  772],

     | 30.00th=[  868], 40.00th=[  956], 50.00th=[ 1048], 60.00th=[ 1160],

     | 70.00th=[ 1320], 80.00th=[ 1608], 90.00th=[ 2416], 95.00th=[ 3536],

     | 99.00th=[ 7456], 99.50th=[10560], 99.90th=[40704], 99.95th=[115200],

     | 99.99th=[407552]

    bw (KB  /s): min=   57, max= 5128, per=6.34%, avg=2549.16, stdev=947.01

    lat (usec) : 500=1.83%, 750=15.90%, 1000=27.12%

    lat (msec) : 2=41.67%, 4=9.52%, 10=3.40%, 20=0.40%, 50=0.07%

    lat (msec) : 100=0.03%, 250=0.04%, 500=0.01%, 750=0.01%, 1000=0.01%

  cpu          : usr=0.28%, sys=2.97%, ctx=1910030, majf=0, minf=525

  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%

     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%

     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%

     issued    : total=r=0/w=1809875/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0

     latency   : target=0, window=0, percentile=100.00%, depth=1

Run status group 0 (all jobs):

  WRITE: io=7069.9MB, aggrb=40219KB/s, minb=40219KB/s, maxb=40219KB/s, mint=180001msec, maxt=180001msec

Read TEST (fio_test_file을 1GB로 16번 읽어서 리포트)

fio --directory=/data --name fio_test_file --direct=1 --rw=randread --bs=16k --size=1G --numjobs=16 --time_based --runtime=180 --group_reporting --norandommap 

fio-2.2.6

Starting 16 processes

Jobs: 16 (f=16): [r(16)] [100.0% done] [55004KB/0KB/0KB /s] [13.8K/0/0 iops] [eta 00m:00s]

fio_test_file: (groupid=0, jobs=16): err= 0: pid=22909: Tue Jan  5 11:38:57 2016

  read : io=8363.7MB, bw=47573KB/s, iops=11893, runt=180027msec

    clat (usec): min=3, max=1297.3K, avg=1341.39, stdev=9923.98

     lat (usec): min=3, max=1297.3K, avg=1341.58, stdev=9923.99

    clat percentiles (usec):

     |  1.00th=[    6],  5.00th=[  326], 10.00th=[  564], 20.00th=[  660],

     | 30.00th=[  724], 40.00th=[  772], 50.00th=[  820], 60.00th=[  868],

     | 70.00th=[  940], 80.00th=[ 1064], 90.00th=[ 1416], 95.00th=[ 3248],

     | 99.00th=[ 8640], 99.50th=[10816], 99.90th=[29568], 99.95th=[171008],

     | 99.99th=[464896]

    bw (KB  /s): min=   25, max= 6752, per=6.45%, avg=3070.37, stdev=1335.97

    lat (usec) : 4=0.01%, 10=3.94%, 20=0.59%, 50=0.14%, 100=0.04%

    lat (usec) : 250=0.04%, 500=2.84%, 750=28.23%, 1000=39.96%

    lat (msec) : 2=17.31%, 4=2.60%, 10=3.64%, 20=0.54%, 50=0.05%

    lat (msec) : 100=0.01%, 250=0.04%, 500=0.02%, 750=0.01%, 1000=0.01%

    lat (msec) : 2000=0.01%

  cpu          : usr=0.31%, sys=2.92%, ctx=2087698, majf=0, minf=515

  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%

     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%

     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%

     issued    : total=r=2141099/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0

     latency   : target=0, window=0, percentile=100.00%, depth=1

Run status group 0 (all jobs):

   READ: io=8363.7MB, aggrb=47572KB/s, minb=47572KB/s, maxb=47572KB/s, mint=180027msec, maxt=180027msec

Disk stats (read/write):

  sda: ios=2039997/0, merge=0/0, ticks=2779724/0, in_queue=2783058, util=100.00%