Take back 30% of Overhead on your Linux file system

//Take back 30% of Overhead on your Linux file system

Take back 30% of Overhead on your Linux file system

MFG Vision’s yieldHUB software processes many multiples of gigabytes of STDF data per day in the case of several high volume customers.  There are periods when the amount of data uploaded to the server is much larger than usual.  In one occasion on the server at one of our large assembly and test customers in Taiwan, processing was delayed by more than 24 hours for a significant number of datalogs.  After review, it turned out that the disks of the storage device couldn’t keep up with the amount of Test Data Man 162488938data being processed and resulted to high I/O wait while the CPUs where running at low utilization.
Linux file systems has a feature that lets you know when was the last time you accessed a file.  While it is a nice feature for normal files, in database files, you don’t need to know this information as this could be meaningless. 
This feature performs an update on the atime (access time) field of the file.  For any database with decent number of select and update/insert queries, the file seeks needed to update the atime field can become a significant overhead, as much as 30% as reported on various web pages.  The atime field update happen even for select queries.
Since you don’t really need to know when was the last time you accessed the database files, there was no need to update the atime field.  So we decided to disable this on the customer’s server by modifying the mount options to use the noatime option.
After this was done, it was obvious that the data processing sped up significantly.  The I/O wait still appeared high but this time the CPU utilization was also much higher then prior to adding the noatime mount option.  This meant that processing the data was now much faster.
Disabling atime update is very simple to implement and should be used if the last access time of the file is not important, such as in a database file system. 
References: 

 

2018-07-18T12:02:53+00:00 December 7th, 2015|Blog|