Intro to Python

as (to be) presented to TCS Programmers' SIG

2002 October November

Special Topic: Run Speed

This topic was only covered in a very cursory way in the actual presentation, because of lack of time. Here are the presentation notes that you guys never got to see. :-)

How fast have people found Python to be compared to other languages?

We will see that there is much disagreement.

Result 1: Colored graph proves that Python is way slower
than Perl, Java, C++

This graph proves it?

These results are kind of old (they obviously date from before the release of Java 2) and we will see more comprehensive benchmark tests below.

Result 0: "Java is generally faster," but maybe not by much, at least in the very old versions here tested, if this tester is to be believed.

Here are the results that inspired a skeptical person to do the testing resulting in "Result 1" on the preceding page.
times are measured in real ("wall clock") seconds, since this is a practical analysis. YMWV.

Test Java Python Comparison

Standard Output 138.85 30.58 Python 4.5X Faster than Java

Hashtable 17.0 8.22 Python 2X Faster than Java

I/O 56.72 47.36 Python 1.2X Faster than Java

List 5.94 14.32 Java 2.4X Faster than Python

Native Methods 2.475 7.92 Java 3.2X Faster than Python

Interpreter Initialisation 0.25 0.04 Python 6.3X Faster than Java

Object Allocation 23.65 211.11 Java 8X Faster than Python

Interpreter Speed 0.43 2.29 Java 5.3X Faster than Python

Again, I must stress that these are approximate times, which are specific to my computer, my Java version (JDK 1.1.7B, blackdown), my operating system (Debian GNU Linux 2.2), my python version (Python 1.5.2) and my idiosyncratic test method, which is not strenuous at all. I repeated the tests 3 times and averaged the results to get the numbers you see here.

Note that the versions of Python and of Java involved are rather old.

Result 2: "The Great Computer Language Shootout" finds Python infinitesimally faster than Perl, significantly slower than C, C++, Java.

This page of benchmarks is far more detailed than the ones above.

This page of benchmarks is far more detailed than the ones above.

SCORES

Language Implementation Score Missing

C gcc 752 0

Ocaml ocaml 751 0

SML mlton 751 0

C++ g++ 743 0

SML smlnj 736 0

Common Lisp cmucl 734 0

Scheme bigloo 730 1

Ocaml ocamlb 718 0

Java java 703 0

Pike pike 647 0

Forth gforth 637 2

Lua lua 615 2

Python python 578 0

Perl perl 577 0

Ruby ruby 546 0

Eiffel se 512 4

Mercury mercury 489 8

Awk mawk 456 6

Haskell ghc 450 5

Lisp rep 424 3

Icon icon 415 7

Tcl tcl 404 3

Javascript njs 397 5

Scheme guile 396 0

Forth bigforth 376 10

Erlang erlang 318 8

Awk gawk 314 6

Emacs Lisp xemacs 311 9

Scheme stalin 207 17

PHP php 197 9

Bash bash 108 14

Languages that compile to native code are in Bold Italics.

(results as of October 24, 2002)

Note that implementations of the "same" language vary radically in performance.

Probably the best feature of "The Great Computer Language Shootout" is that to decide which language is 'best', the implementor (Doug Bagley) provides the "Completely Random and Arbitrary Point System!, or CRAPS![TM]" which lets you weight different factors to put your favorite language nearer the top of the rankings. :-)

I, however, for whatever it is worth, am using Doug's default weights for ranking.

Doug Bagley also has a very good list of links (at the bottom of the main shootout page.)

Result 3: "The Great Computer Language Shootout" ported to Win32 by Aldo Calpini.

Results are somewhat different, but not all that much. Now Python is slightly slower than Perl.

SCORES

Language Implementation Score Missing Failing Avg.Score

C mingw32 648 1 3 30.86

Delphi delphi 365 12 1 30.42

C vc 664 2 1 30.18

Ocaml ocaml 662 0 3 30.09

C bcc 653 2 1 29.68

C lcc 647 3 0 29.41

Ocaml ocamlb 641 0 3 29.14

Mercury mercury 492 7 1 28.94

C gcc 664 0 2 28.87

Lua lua 661 2 0 28.74

SML smlnj 483 0 8 28.41

Common Lisp poplisp 392 0 11 28.00

C++ vc++ 419 6 4 27.93

Eiffel se 497 5 2 27.61

Pascal fpascal 411 8 2 27.40

Pike pike 683 0 0 27.32

Perl perl 652 0 1 27.17

Parrot parrot 214 15 2 26.75

Perl cygperl 614 0 2 26.70

Oz oz 346 10 2 26.62

Python python 639 0 1 26.63

Forth gforth 609 1 1 26.48

ElastiC elastic 157 16 3 26.17

Awk mawk 471 6 1 26.17

Pascal vpascal 444 8 0 26.12

Forth bigforth 287 9 5 26.09

Java java 594 0 2 25.83

C# csharp 411 9 0 25.69

Pliant pliant 509 4 1 25.45

Awk awka 479 6 0 25.21

Icon icon 400 9 0 25.00

Haskell ghc 366 4 6 24.40

Erlang erlang 294 6 6 22.62

Tcl tcl 440 3 2 22.00

Modula-3 modula3 109 20 0 21.80

Scheme guile 459 0 4 21.86

Ruby ruby 405 0 6 21.32

PHP php 298 9 1 19.87

Awk gawk 358 6 0 18.84

JavaScript jscript 214 10 2 16.46

VBScript vbscript 168 5 8 14.00

Rexx rexx 178 7 5 13.69

Simula cim 37 22 0 12.33

Languages that compile to native code are in Bold Italics.

(results as of October 24, 2002)

And now the anticlimax...

My Win32 (Windows 98) example test results

I wrote up a fairly simple program calling functions to generate a list of pseudo-random integers using a fairly simple arithmetic algorithm, sort the list, and print it to standard output. I wrote implementations:

The C/C++ were in Microsoft C++ 5.0, which I happened to have around and be messing with.

All of these implementations were console applications.

The programs are not perfectly equivalent, but are approximately equivalent.

I tried to test them under similar levels of system load, and measured the actual "wall clock" time elapsed rather than trusting the development tools' profilers.

The time test results are kind of crude and approximate.

Actual test output

Python:

996, 997, 997, 997, 997, 997, 997, 997, 997, 997, 997, 997, 997, 997, 998, 998,
998, 998, 998, 998, 998, 998, 998, 998, 998, 998, 998, 998, 998, 998, 999, 999,
999, 999, 999, 999, 999, 999, 999, 999, 999, 999, 999, 999, 1000, 1000, 1000, 10
00, 1000, 1000, 1000, 1000, 1000, 1000, 1000, 1000, 1001, 1001, 1001, 1001, 1001
, 1001, 1001, 1001, 1001, 1001, 1001, 1002, 1002, 1002, 1002, 1002, 1002, 1002,
1002, 1002, 1002, 1002, 1003, 1003, 1003, 1003, 1003, 1003, 1003, 1003, 1003, 10
03, 1003, 1003, 1003, 1003, 1003, 1004, 1004, 1004, 1004, 1004, 1004, 1004, 1004
, 1004, 1004, 1004, 1005, 1005, 1005, 1005, 1005, 1005, 1005, 1005, 1005, 1005,
1006, 1006, 1006, 1006, 1006, 1006, 1006, 1006, 1006, 1006, 1007, 1007, 1007, 10
07, 1007, 1007, 1007, 1007, 1007, 1008, 1008, 1008, 1008, 1008, 1008, 1008, 1008
, 1008, 1008, 1008, 1008, 1008, 1009, 1009, 1009, 1009, 1009, 1009, 1009, 1010,
1010, 1010, 1010, 1010, 1010, 1010, 1010, 1011, 1011, 1011, 1011, 1011, 1011, 10
11, 1011, 1012, 1012, 1012, 1012, 1012, 1012, 1012, 1012, 1012, 1013, 1013, 1013
, 1013, 1013, 1013, 1013, 1013, 1013, 1013, 1013, 1014, 1014, 1014, 1014, 1014,
1014, 1015, 1015, 1015, 1015, 1015, 1015, 1016, 1016, 1016, 1016, 1016, 1016, 10
16, 1017, 1017, 1017, 1017, 1017, 1017, 1017, 1018, 1018, 1018, 1018, 1018, 1018
, 1018, 1019, 1019, 1019, 1020, 1020, 1020, 1020, 1021, 1021, 1021, 1021, 1021,
1022, 1022, 1022, 1022, 1022, 1022, 1023, 1023, 1023, 1023, 1024, 1024, 1024, 10
25, 1025, 1026, 1026, 1026, 1027, 1027, 1027, 1027, 1028, 1028, 1029, 1031, 1031
, 1032, 1032, 1034, 1034]
times : [4.2738002648384992, 4.9026140230308926, 4.9026458706984695, 5.288777049
5650281, 27.185435558758947]
EXITING FUNCTION num_list_test(50000)

Perl: (Note that the math turns out slightly different.)

 996, 996, 996, 996, 996, 996, 996, 997, 997, 997, 997, 997, 997, 997, 997, 997,
 997, 997, 997, 998, 998, 998, 998, 998, 998, 998, 998, 998, 998, 998, 998, 999,
 999, 999, 999, 999, 999, 999, 999, 999, 999, 999, 999, 999, 1000, 1000, 1000, 1
000, 1000, 1000, 1000, 1000, 1000, 1000, 1000, 1000, 1000, 1000, 1000, 1000, 100
1, 1001, 1001, 1001, 1001, 1001, 1001, 1001, 1001, 1001, 1001, 1001, 1002, 1002,
 1002, 1002, 1002, 1002, 1002, 1002, 1002, 1002, 1003, 1003, 1003, 1003, 1003, 1
003, 1003, 1003, 1003, 1004, 1004, 1004, 1004, 1004, 1004, 1004, 1004, 1004, 100
4, 1004, 1005, 1005, 1005, 1005, 1005, 1005, 1005, 1005, 1005, 1005, 1005, 1005,
 1005, 1005, 1006, 1006, 1006, 1006, 1006, 1006, 1006, 1006, 1006, 1006, 1007, 1
007, 1007, 1007, 1007, 1007, 1007, 1007, 1007, 1007, 1008, 1008, 1008, 1008, 100
8, 1008, 1008, 1008, 1009, 1009, 1009, 1009, 1009, 1009, 1009, 1009, 1009, 1010,
 1010, 1010, 1010, 1010, 1010, 1010, 1010, 1010, 1010, 1011, 1011, 1011, 1011, 1
011, 1011, 1011, 1011, 1012, 1012, 1012, 1012, 1012, 1012, 1012, 1013, 1013, 101
3, 1013, 1013, 1013, 1013, 1014, 1014, 1014, 1014, 1014, 1014, 1014, 1014, 1014,
 1015, 1015, 1015, 1015, 1015, 1015, 1015, 1015, 1016, 1016, 1016, 1016, 1016, 1
016, 1017, 1017, 1017, 1017, 1017, 1017, 1018, 1018, 1018, 1018, 1018, 1019, 101
9, 1019, 1019, 1019, 1019, 1020, 1020, 1020, 1020, 1021, 1021, 1021, 1021, 1022,
 1022, 1022, 1022, 1023, 1023, 1023, 1023, 1023, 1024, 1024, 1024, 1024, 1024, 1
025, 1025, 1026, 1026, 1027, 1027, 1027, 1028, 1028, 1029, 1029, 1029, 1032, 103
2, 1033, 1033, 1034, 1035, 1039
times : 1035361693, 1035361695, 1035361695, 1035361697, 1035361720
EXITING FUNCTION num_list_test(50000)

C (totally integer implementation):

 1018, 1019, 1019, 1019, 1020, 1020, 1020, 1020, 1021, 1021, 1021, 1021, 1021, 1
022, 1022, 1022, 1022, 1022, 1022, 1023, 1023, 1023, 1023, 1024, 1024, 1024, 102
5, 1025, 1026, 1026, 1026, 1027, 1027, 1027, 1027, 1028, 1028, 1029, 1031, 1031,
 1032, 1032, 1034, 1034
5497.005000 secs, 5497.042000 secs, 5497.167000 secs, 5514.164000 secs, (end)
EXITING FUNCTION num_list_test(50000)

C (totally floating-point implementation as much as conveniently supported):

0, 1026.000000, 1027.000000, 1027.000000, 1027.000000, 1027.000000, 1028.000000,
 1028.000000, 1029.000000, 1031.000000, 1031.000000, 1032.000000, 1032.000000, 1
034.000000, 1034.000000
6086.704000 secs, 6086.752000 secs, 6087.088000 secs, 6153.392000 secs, (end)
EXITING FUNCTION num_list_test(50000)

C++ with STL, floating-point: Failed: list::sort() library function lost data.

 1011, 1011, 1011, 1012, 1012, 1012, 1013, 1013, 1013, 1013, 1013, 1014, 1014, 1
014, 1015, 1015, 1016, 1016, 1017, 1018, 1018, 1018, 1019, 1020, 1020, 1021, 102
1, 1022, 1022, 1022, 1023, 1023, 1023, 1024, 1025, 1027, 1028, 1031, 1032
times : [15369.502, 15369.692, 15369.692, 15370.508, 15380.796]
EXITING FUNCTION num_list_test(50000)

Python:

1027, 1028, 1028, 1028, 1028, 1028, 1029, 1030, 1031, 1031, 1031, 1031, 1032, 1 032, 1032, 1032, 1033, 1033, 1034, 1034, 1038] times : [28.522181062371143, 29.787491409510718, 29.787523257178297, 30.61960559 1779948, 65.907701268878128] EXITING FUNCTION num_list_test(100000) ... 1027, 1028, 1028, 1028, 1028, 1028, 1029, 1030, 1031, 1031, 1031, 1031, 1032, 1 032, 1032, 1032, 1033, 1033, 1034, 1034, 1038] times : [28.625230895589937, 29.859954910407481, 29.859987596171575, 30.68954223 1683401, 66.040143146884802] EXITING FUNCTION num_list_test(100000)

Perl:

1028, 1028, 1029, 1029, 1029, 1030, 1030, 1030, 1030, 1030, 1031, 1031, 1031, 10 32, 1032, 1033, 1033, 1034, 1035, 1037, 1040 times : 1035363961, 1035363980, 1035363981, 1035363986, 1035364030 EXITING FUNCTION num_list_test(100000) ... 1028, 1028, 1029, 1029, 1029, 1030, 1030, 1030, 1030, 1030, 1031, 1031, 1031, 10 32, 1032, 1033, 1033, 1034, 1035, 1037, 1040 times : 1035364167, 1035364187, 1035364188, 1035364193, 1035364235 EXITING FUNCTION num_list_test(100000)

C (totally integer implementation):

1027, 1028, 1028, 1028, 1028, 1028, 1029, 1030, 1031, 1031, 1031, 1031, 1032, 10 32, 1032, 1032, 1033, 1033, 1034, 1034, 1038 5286.392000 secs, 5286.487000 secs, 5286.997000 secs, 5316.153000 secs, (end) EXITING FUNCTION num_list_test(100000)

C (floating-point implementation):

1027.000000, 1027.000000, 1028.000000, 1028.000000, 1028.000000, 1028.000000, 10 28.000000, 1029.000000, 1030.000000, 1031.000000, 1031.000000, 1031.000000, 1031 .000000, 1032.000000, 1032.000000, 1032.000000, 1032.000000, 1033.000000, 1033.0 00000, 1034.000000, 1034.000000, 1038.000000 1727.592000 secs, 1727.704000 secs, 1728.331000 secs, 1918.228000 secs, (end) EXITING FUNCTION num_list_test(100000)

Generate-sort-print execution time test.
Data set size: 50000

(language) calculate and build list of pseudo-random numbers using simple arithmetic algorithm sort (using built-in sort function) print the numbers to stdout

Python 0.6 0.4 ~16 (15-22)

Perl 5.004 ~2 (2-3) ~2 (1-2) ~23?

Perl 5.6 ~1? (1-2) ~1? (1-3) ~16? (8-24)

C (integer implementation) ~0.04 ~0.1 ~17

C (floating-point implementation) ~0.05 ~0.3 ~60

high-level C++ (floating-point, Microsoft STL-based implementation) ~0.2 (failed--lost data) (N/A)

Generate-sort-print execution time test.
Data set size: 100000

(language) calculate and build list of pseudo-random numbers using simple arithmetic algorithm sort (using built-in sort function) print the numbers to stdout

Python 1 0.8 22? (21-~36)

Perl 5.004 20+ (21-28) ~5 ~43?

Perl 5.6 ~2 (~1-25) ~3 (3-6) ~20 (18-27)

C (integer implementation) ~0.1 ~0.5 ~30

C (floating-point implementation) ~0.1 ~0.6 ~100

Comments on my time test results

All these times are approximate, and could probably be tweaked depending on many factors.

A few examples:

In IDLE (open-source GUI for Python development), performance was worse by maybe 2x - 3x.
In integer low-level C version, I used subtraction for comparison function to support qsort() call.
In floating-point C/C++ versions, when printing I could have converted floating-point to int, or at least limited the precision to save cycles.
Machine load is never perfectly consistent; times can easily vary by 5%-15%, and in some cases by (much) greater amounts.

Python vs. Perl

Python is faster than Perl in the majority of the arbitrarily chosen tests?! This result is contrary to the traditional 'conventional wisdom' of many.

Understandably, I sat down with a watch to check that Python was really giving me the actual total system time elapsed, not just time allotted to its process or some other noncomparable number like that. But it turned out I really was getting an appropriate number for total system time elapsed.

We can "blame" the Perl results on many partially extenuating factors, but some would be at least partly to Python's credit.

You didn't test enough things / the right things! (A reasonable concern, although to be fair, my personal time budget was limited.)
"Who cares what happens on a Windows machine anyway?" (If I want a broadly cross-platform development toolkit, I do.)
Maybe the Windows port is not as good as the other versions of Perl? (Quite possible IMHO)
Maybe Perl is better/more optimized for Linux or *NIX, and/or the C/C++ compiler used to build it is better optimized for those OSes?
Maybe a simpler language is easier to broadly optimize across many different platforms?

In my opinion, Perl and C++ code gave more weird bugs / surprising behavior during development and debugging, especially if I allow for my relative levels of experience in the various languages. But the worst C++ bugs we can blame on Microsoft. :-)

Python vs. C/C++

The C/C++ code could probably be modified to better optimize it.

The low-level C code as written was definitely more work to debug relative to the programmer's level of proficiency, than the Python code was. Using a good C/C++ development toolkit (e.g. container libraries) would have improved this considerably.

In my opinion, the higher-level C++ code's poor support from the compiler was not representative of a good implementation of C++ and of STL. It is not entirely surprising that an old Microsoft implementation of STL is not likely to be up to par. (rather as their templates support was crap compared to at least one leading compiler for Windows back when templates were relatively new)

You might want to go back to the main presentation.

Generate-sort-print execution time test. Data set size: 50000
(language)	calculate and build list of pseudo-random numbers using simple arithmetic algorithm	sort (using built-in sort function)	print the numbers to stdout
Python	0.6	0.4	~16 (15-22)
Perl 5.004	~2 (2-3)	~2 (1-2)	~23?
Perl 5.6	~1? (1-2)	~1? (1-3)	~16? (8-24)
C (integer implementation)	~0.04	~0.1	~17
C (floating-point implementation)	~0.05	~0.3	~60
high-level C++ (floating-point, Microsoft STL-based implementation)	~0.2	(failed--lost data)	(N/A)