Abstract

Log files in complex systems quickly grow into huge sizes. Often, they must be kept for a long period of time. For the convenience and storage economy, log files should be compressed. However, most of the available log file compression tools use a general-purpose algorithms, which do not take advantage of redundancy specific log files. The objective of this paper is to use Length Index Preserving Transformation (LIPT) technique to transform the log files and reduces the significant amount of redundant characters. The applied transformed file in data compression tool will effectively reduce storage space. It has been observed that LIPT is an effective way to transform log files for size reduction and the file size get reduced to the average of 44% before the compression. During the process of transformation the redundant character gets reduced.

Keyword

Compression, LIPT, Reduction, Transformation.

Cite this article

Refference

[1][1]http://techterms.com/definition/logfile. Accessed 01 October 2015.

[2][2]Sree PK, Babu IR. FELFCNCA: fast & efficient log file compression using non linear cellular automata classifier. International Journal on Communications. 2012; 1(1): 7-11.

[3][3]Singh NK, Tomar DS, Roy BN. An approach to understand the end user behavior through log analysis. International Journal of Computer Applications. 2010; 5(11):9-13.

[4][4]Rácz B, Lukács A. High density compression of log files. In proceedings of data compression conference 2004 (p. 557). IEEE.

[5][5]Balakrishnan R, Sahoo RK. Lossless compression for large scale cluster logs. In parallel and distributed processing symposium (IPDPS) 2006 (pp. 1-7). IEEE.

[6][6]Skibiński P, Swacha J. Fast and efficient log file compression. In proceedings of 11th east-European conference on advances in databases and information systems (ADBIS) 2007 (pp. 56-69).

[7][7]Grabowski S, Deorowicz S. Web log compression. Automatyka/Akademia Górniczo-Hutnicza im. Stanisława Staszica w Krakowie. 2007;11(3):417-24.

[8][8]Feng C, Wang H, Lu N, Chen T, He H, Lu Y, et al. Log-transformation and its implications for data analysis. Shanghai archives of psychiatry. 2014; 26(2):105-9.

[9][9]Fageeri SO, Ahmad R. An efficient log file analysis algorithm using binary-based data structure. Procedia-Social and Behavioral Sciences. 2014;129:518-26.

[10][10]Anitha J, Babu M. ETL work flow for extract transform loading. International Journal of Computer Science and Mobile Computing. 2014;3(6):610-7.

[11][11]Awan FS, Mukherjee A. LIPT: a lossless text transform to improve compression. In international conference on information technology: coding and computing 2001 (pp. 452-60). IEEE.

[12][12]http://shodhganga.inflibnet.ac.in/bitstream/10603/28063/7/07_chapter1.pdf. Accessed 30 October 2015.