Oracle
and linux I/O Scheduler
Part 3: Tuning the cfq scheduler
Starting from the previous two papers and thanks to Jens Axboe advices
I decided to start several test trying to tune the cfq scheduler.
The database has been recreated as well as the TPCC schema. For this
reason I performed a couple of benchmarch with the deadline scheduler
as a baseline for the tests on cfq. These have been made changing one
(or two) parameter at one time and the comparing the results in the
usual table.
Important note: The db block
size has been shrinked from 8K to 4K since I wished to compare the
performance of a transactional workload having my DB block size
identical to the filesystem blocksize.
The cfq default values are (the yellow parameters are common to all the
schedulers):
slice_idle
|
8
|
How long a sync slice is allowed
to idle |
slice_async_rq
|
2
|
How many requests an async disk
slice lasts |
slice_async
|
40
|
How many msec an async disk
slice lasts |
slice_sync
|
100
|
How many msec a sync disk slice
lasts |
back_seek_penalty
|
2
|
penalty of a backwards seek |
back_seek_max
|
16384 |
maximum backwards seek |
fifo_expire_async
|
248 |
fifo timeout for async requests |
fifo_expire_sync
|
124 |
fifo timeout for sync requests |
queued
|
8
|
minimum request allocate limit
per-queue |
quantum
|
4
|
max number of requests moved in
one round of service |
max_sectors_kb
|
64
|
max
transfer size allowed for this queue |
max_hw_sectors_kb
|
64
|
Max
transfer size that the driver and/or device can handle |
read_ahead_kb
|
512
|
Size
of read-ahead window |
nr_requests
|
128
|
Maximum
number of requests allowed in a queue |
The tested configuration:
0. The baseline test with the deadline cheduler (and its default
parameters).
a. datafile with cfq and
echo 0 >
/sys/block/sdb/queue/iosched/slice_idle
b. The above configuration plus:
echo 64 >
/sys/block/sdb/queue/iosched/max_depth
c. configuration b with slice_idle at default:
echo 1 >
/sys/block/sdb/queue/iosched/slice_idle
d. configuration a and b plus some more tuning on the queues:
echo 0 >
/sys/block/sdb/queue/iosched/slice_idle
echo 64 > /sys/block/sdb/queue/iosched/max_depth
echo 32 > /sys/block/sdb/queue/iosched/queued
echo 64 > /sys/block/sdb/queue/iosched/quantum
e. redolog and datafiles with cfq tuned and queues at default:
echo 0 >
/sys/block/sd[b,c]/queue/iosched/slice_idle
echo 64 > /sys/block/sd[b,c]/queue/iosched/max_depth
echo 8 > /sys/block/sd[b,c]/queue/iosched/queued
echo 4 > /sys/block/sd[b,c]/queue/iosched/quantum
(queued and quantum are the default values).
f. previous configuration with queues increased:
echo 0 >
/sys/block/sd[b,c]/queue/iosched/slice_idle
echo 64 > /sys/block/sd[b,c]/queue/iosched/max_depth
echo 32 > /sys/block/sd[b,c]/queue/iosched/queued
echo 64 > /sys/block/sd[b,c]/queue/iosched/quantum
Results:
For any scheduler you can see an AWR report following the below links:
|
transaction per second
|
log file sync %
|
user calls
|
physical reads
|
physical writes
|
test 0
|
53.34 |
.7 |
248.87 |
310.22 |
109.15 |
test a
|
37.41 |
.6 |
142.70 |
223.40 |
71.15 |
| test b |
52.60 |
.8 |
246.77 |
312.64 |
106.40 |
test c
|
31.51 |
.3 |
147.38 |
187.46 |
64.13 |
test d
|
52.26 |
.3 |
242.08 |
313.31 |
108.89 |
test e
|
53.24 |
.8 |
246.53 |
313.00 |
105.62 |
test f
|
52.05 |
.8
|
242.55 |
313.96 |
106.04 |
The cfq scheduler performs a lot better in any test with the specified
parameters modified from their default values.
However it seems the redologs gives better results if placed on a
device with the deadline scheduler activated.
Note: when the redologs are on cfq the "log file sync" is lower than
when placed on deadline. The overall performance, instead, shows better
reults in the first cases (with deadline).
Looking at the tpm graph during the a and b tests shows that the
transaction per minute in b is more constant (the values of the table
are the mean values).
The same can be seen in c and d.
After the above test I decided to reset the whole system regenerating
the TPC-C schema and rerunning all the tests I already performed to be
sure the results are coherent plus several new trying to further tune
cfq and deadline schedulers.
Contact information:
fabrizio.magni _at_ gmail.com