Productive Small-scale Data Processing using Python

doi:10.12972/ksmer.2014.51.5.705

All Issue

2014 Vol.51, Issue 5 Preview Page Next Page

Technical Report

Productive Small-scale Data Processing using Python 파이썬을 이용한 생산적인 소규모 자료처리: 하완수
Wansoo Ha

31 October 2014. pp. 705-714

PDF

Abstract

Python is an object oriented programming language, similar to Perl and Ruby. It supports a dynamic typing and operates using a python interpreter. Static typing languages such as Fortran and C emphasize efficient execution program. On the other hand, Python’s design philosophy values productivity of programmers. In general, a Python version of an algorithm is shorter and slower than its Fortran or C versions. Recent development of numerical libraries shortened the runtime of Python numerical programs and many scientists and engineers adopt Python as their programming language. In this study, I compared Python and Fortran programming languages by developing functions of simple algorithms used in seismic data processing. I tried to make Python programs efficient by exploiting numerical libraries. Benchmarking examples show that small-scale data processing programming using Python and Numpy library can be more efficient than that using Fortran language.

Keywords

Data processing

Python

Productivity

Efficiency

파이썬은 펄, 루비 등과 같은 객체지향 프로그래밍 언어로, 해석기를 통해 작동하며 동적 자료형을 지원한다. 계산 속도를 중시하여 정적 자료형을 이용하는 포트란 프로그래밍에서와 달리 파이썬 프로그래밍에서는 개발자의 생산성을 더 중시한다. 일반적인 파이썬 프로그램의 경우 동일한 기능의 포트란 프로그램에 비해 개발 시간은 짧지만 실행 시간은 길다. 최근 수치 계산 라이브러리들이 발전되면서 파이썬 프로그램의 실행 시간이 짧아지고 있고, 과학 및 공학 계산에 파이썬을 이용하는 사례가 증가하고 있다. 본 연구에서는 탄성파 자료처리에 사용되는 간단한 알고리즘들을 대상으로 수치 라이브러리들을 통해 파이썬 프로그램의 실행 속도를 향상시키는 방법을 살펴본다. 예제들을 통해 프로그램 작성 시간을 고려하면 소규모 자료처리의 경우 포트란 대신 파이썬과 Numpy 라이브러리를 이용하는 것이 더 효율적임을 알 수 있다.

키워드

자료처리

파이썬

생산성

효율성

References

Beazley, D.M., 2009, Python essential reference, 4th ed., Developer’s Library, pp. 5-24.
Billette, F.J. and Brandsberg-Dahl, S., 2005, “The 2004 BP velocity benchmark,” 67th Annual Meeting, EAGE, Madrid, Spain, June 13-16, p. 8035.
BLAS, 2014.8.28., http://www.netlib.org/blas.
Fleming, P.J. and Wallace, J.J., 1986, “How not to lie with statistics: The correct way to summarize benchmark results,” Communications of the ACM, Vol. 29, No. 3, pp. 218-221.
IEEE Spectrum, 2014.8.28., http://spectrum.ieee.org/static/ interactive-the-top-programming-languages.
IPython, 2014.8.28., http://ipython.org.
Janert, P.K., 2010, Data analysis with open source tools, O’Reilly, p. 439.
Julia Programming Language, 2014.7.22, http://julialang.org.
Kreyszig, E., 2011, Advanced engineering mathematics, 10th ed., John Wiley & Sons, Inc, p. 263.
Lanaro, G., 2013, Python high performance programming, Packt Publishing, pp. 49-70.
LAPACK, 2014.8.28., http://www.netlib.org/lapack.
LLVM, 2014.7.22, http://llvm.org.
Madagascar, 2014.7.22, http://www.ahay.org.
Mankiw, N.G., 2011, Principles of economics, 6th ed., Cengage Learning, p. 12.
Matlab, 2014.7.22, http://www.mathworks.co.kr/products/ matlab.
McKinney, W., 2012, Python for data analysis, O’Reilly, pp. 3-6.
Netlib, 2014.7.22, http://www.netlib.org.
Numba, 2014.7.22, http://numba.pydata.org.
Numexpr, 2014.7.22, http://github.com/pydata/numexpr.
Numpy, 2014.7.22, http://www.numpy.org.
Prechelt, L., 2000, “An empirical comparison of seven programming languages,” IEEE Computer, Vol. 33, No. 10, pp. 23-29.
Press, W.H., Teukolsky, S.A., Vetterling, W.T., Flannery, B.P. and Metcalf, M., 1996, Numerical Recipes in Fortran 90: Vol. 2, Cambridge University Press, p. 1.
PyLab, 2014.8.28., http://wiki.scipy.org/PyLab.
PyPy, 2014.7.22, http://pypy.org.
Python, 2014.7.22, http://www.python.org.
SciPy, 2014.7.22, http://www.scipy.org.
Seismic Unix, 2014.7.22, http://www.cwp.mines.edu/cwpcodes.
The Python Language Reference, 2014.7.22, http://docs. python.org.
The R project, 2014.7.22, http://www.r-project.org.
Vaingast, S., 2009, Beginning python visualization, Apress, pp. 249-284.
Versteeg, R., 1994, “The Marmousi experience: velocity model determination on a synthetic complex data set,” Leading Edge, Vol. 13, pp. 927-936.
Weinberg, G.M., 1998, The Psychology of computer programming: Silver anniversary edition, Weinberg & Weinberg, p. 63.

Information

Publisher :The Korean Society of Mineral and Energy Resources Engineers
Publisher(Ko) :한국자원공학회
Journal Title :Journal of the Korean Society of Mineral and Energy Resources Engineers
Journal Title(Ko) :한국자원공학회지
Volume : 51
No :5
Pages :705-714
DOI :https://doi.org/10.12972/ksmer.2014.51.5.705

[1] Beazley, D.M., 2009, Python essential reference, 4th ed., Developer’s Library, pp. 5-24.

[2] Billette, F.J. and Brandsberg-Dahl, S., 2005, “The 2004 BP velocity benchmark,” 67th Annual Meeting, EAGE, Madrid, Spain, June 13-16, p. 8035.

[3] BLAS, 2014.8.28., http://www.netlib.org/blas.

[4] Fleming, P.J. and Wallace, J.J., 1986, “How not to lie with statistics: The correct way to summarize benchmark results,” Communications of the ACM, Vol. 29, No. 3, pp. 218-221.

[5] IEEE Spectrum, 2014.8.28., http://spectrum.ieee.org/static/ interactive-the-top-programming-languages.

[6] IPython, 2014.8.28., http://ipython.org.

[7] Janert, P.K., 2010, Data analysis with open source tools, O’Reilly, p. 439.

[8] Julia Programming Language, 2014.7.22, http://julialang.org.

[9] Kreyszig, E., 2011, Advanced engineering mathematics, 10th ed., John Wiley & Sons, Inc, p. 263.

[10] Lanaro, G., 2013, Python high performance programming, Packt Publishing, pp. 49-70.

[11] LAPACK, 2014.8.28., http://www.netlib.org/lapack.

[12] LLVM, 2014.7.22, http://llvm.org.

[13] Madagascar, 2014.7.22, http://www.ahay.org.

[14] Mankiw, N.G., 2011, Principles of economics, 6th ed., Cengage Learning, p. 12.

[15] Matlab, 2014.7.22, http://www.mathworks.co.kr/products/ matlab.

[16] McKinney, W., 2012, Python for data analysis, O’Reilly, pp. 3-6.

[17] Netlib, 2014.7.22, http://www.netlib.org.

[18] Numba, 2014.7.22, http://numba.pydata.org.

[19] Numexpr, 2014.7.22, http://github.com/pydata/numexpr.

[20] Numpy, 2014.7.22, http://www.numpy.org.

[21] Prechelt, L., 2000, “An empirical comparison of seven programming languages,” IEEE Computer, Vol. 33, No. 10, pp. 23-29.

[22] Press, W.H., Teukolsky, S.A., Vetterling, W.T., Flannery, B.P. and Metcalf, M., 1996, Numerical Recipes in Fortran 90: Vol. 2, Cambridge University Press, p. 1.

[23] PyLab, 2014.8.28., http://wiki.scipy.org/PyLab.

[24] PyPy, 2014.7.22, http://pypy.org.

[25] Python, 2014.7.22, http://www.python.org.

[26] SciPy, 2014.7.22, http://www.scipy.org.

[27] Seismic Unix, 2014.7.22, http://www.cwp.mines.edu/cwpcodes.

[28] The Python Language Reference, 2014.7.22, http://docs. python.org.

[29] The R project, 2014.7.22, http://www.r-project.org.

[30] Vaingast, S., 2009, Beginning python visualization, Apress, pp. 249-284.

[31] Versteeg, R., 1994, “The Marmousi experience: velocity model determination on a synthetic complex data set,” Leading Edge, Vol. 13, pp. 927-936.

[32] Weinberg, G.M., 1998, The Psychology of computer programming: Silver anniversary edition, Weinberg & Weinberg, p. 63.

Journal of the Korean Society of Mineral and Energy Resources Engineers ISSN:2288-0291(Print) 2288-2790(Online) 한국자원공학회지

All Issue