Mathematica 3.0
Reviewed by Kayll Lake for Physics World.
The booklet "A First Look At Mathematica 3.0" starts with the claim that "Mathematica is the world's only fully integrated environment for technical computing." It then states that "It is often said that the release of Mathematica marked the beginning of modern technical computing." Whereas a defense might be found for the term "fully integrated", I find it difficult to believe that any informed person would swallow the second claim. When Mathematica was introduced both Macsyma and Reduce were highly developed systems with a significant user base. What Mathematica has done is to usher in a new era of fully commercialized technical software.

For those who are not familiar with Mathematica, it is an all-purpose computer algebra software package that uses a high-level programming language. It can, for example, be used to integrate and differentiate functions to any degree of precision, evaluate advanced mathematical expressions and perform three-dimensional visualizations. This new version of Mathematica has been five years in coming and, as we would expect, it offers several new features. These include support for fully editable, typeset functions that are entered into an interactive notebook. A user can, for example, create an integral function that looks just like it would on a printed page by selecting icons and symbols from tool palettes. If the integral does not have limits, Mathematica gives a symbolic result; if it does have limits, the answer is quoted as a number. The list of other new features is long. There are changes in numerical and algebraic computation, mathematical functions, graphics programming and the core system. The input and output, the notebook and the system interface are also different from earlier versions, and the series of "standard add-on packages" has also been revised. Whereas I encountered problems with some of the new features, Mathematica 3.0 is a "new shoe" and we can expect most teething problems to be resolved in short order. The ease of use of the system is, to a certain extent, a matter of taste. For example, I doubt that a working mathematician will be too enthusiastic about pointing and clicking at palettes to write out equations. A beginner might, but they might also be intimidated by the complexity of the complete interface.

In addition to Mathematica and a 1400-page guidebook, which is part of the help system, there is a remarkable array of supplementary material. This includes applications from Wolfram Research, third-party applications, hundreds of books, and MATH Source which is described as "a vast electronic library of Mathematica material, with over 100,000 pages of immediately accessible Mathematica programs, documents, examples, and more." Whereas the shear volume of this material is impressive, I cannot say that everything I looked at impressed me. Overstatements are common. For example, Wolfram's Scientific Astronomer claims to be "The World's Most Sophisticated General Astronomy System". From the books I looked at I would have to say that not all are of high quality. Moreover, MATH Source cannot be taken as a definitive up to date source of Mathematica expertise. I found examples which underestimate Mathematica's capabilities, examples which are years out of date, and examples with incomplete notebooks.

Of particular interest to me is the claim that Mathematica 3.0 offers "Faster execution speed and lower memory usage for typical kernel operations." Whereas the speed of a system is not of central interest to every user, it is a prime concern for many researchers. The change in the relative speed of a system between versions is, I think, of interest to all. Mathematica 3.0 currently runs under Microsoft Windows, MacOS, Linux, UNIX and NEXTSTEP version 3.0 or higher. The version of Mathematica 3.0 I tested was designed for Microsoft Windows 95, and the version I had to compare it to was designed for earlier incarnations of Microsoft Windows. To accommodate this, I re partitioned my hard drive and set up a dual boot Windows 3.11/ Windows 95 environment by offering up an identical copy of my first partition to the Windows 95 (4.00.950) upgrade procedure. Mathematica 2.2 was installed in Windows 3.11, and in Windows 95 through the upgrade. Mathematica 3.0 was installed into Windows 95 from a CD. The installation is effortless. As described below, I ran tests of Mathematica 2.2 under Windows 3.11 and Windows 95, and tests of Mathematica 3.0 under Windows 95. (The first system used was an Intel Pentium 133 MHz, on an Asus PCI/E-P54NP4, with 512 K cache and 48 MB RAM. The disc was a Seagate ST11950N/ND SCSI-2, and video card an ATI graphics Pro Turbo. The tests were re executed with an Intel Pentium Pro as described below.)

Timings for various kinds of Mathematica functions and programs run on different computer configurations have been collected by Stefan Steinhaus of the University of Frankfurt in Germany, and by Karl Unterkofler of the Technical University in Graz, Austria. These now standard tests were used to produce the first two sets of tables shown below. In each case three separate runs were used. In Windows 3.11 the runs were from separate boots from DOS, and for Windows 95 the runs were from separate (warm) boots of the computer. These tests were supplemented by a series of computationally more intensive tests with GRTensorM. For these tests MaxMemoryUsed is reported as well as TimeUsed. Load time can be distinguished from actual computation time for these tests.

The tests were re executed with an Intel Pentium Pro CPU rated at 150 MHz, on an i440FX Amptron mainboard with 32 MB EDO RAM. The disc was an IBM 2.1GB IDE, and video card a Matrox Mystique. The system has a MMA 3.0 Benchmark of 0.75 according to the test by Karl Unterkofler. (This benchmark for the original system is 0.37.) The results at 150 MHz (f = 60 MHz , 2.5x) are reported in red. Clocked at 166 Mhz (f = 66 MHz , 2.5x) the MMA 3.0 Benchmark is 0.84 and the results of the tests are reported in green. Clocked at 180 MHz (f = 60 MHz , 3x) the MMA 3.0 Benchmark is 0.90 and the results of the tests are reported in brown. The chip would not run at 200 MHz. For the tests with the Pentium Pro both Mathematica 2.2 and 3.0 were installed into Microsoft Windows 95 (4.00.950 B).

Conclusions:


Note the following:
Some End Notes on these timings are given following the tables.

Standard Mathematica Benchmarks

Mathematica timing tests by Stefan Steinhaus.
(Stefan.ma)

OS Ver. 1 2 3 4 5
3.11 2.20.05+0.01-05.27+0-0.112.58+0+0.052.31+0-0.6 00.88+0.11+0.11
95 2.2 0.06+0-0.015.272.64-0.06-0.064.23-1.86-1.920.93 +0.06+0.06
95 3 0.06 9.17+ 0-0.063.18+0+0.019.451.26
95 2.2 0+0.05+0.051.651.711.48-0.05+0.050.33
95 3 0+0.06+0 4.012.08-0.05-0.117.41-2.08-3.180.61
95 2.2 0+0+0.051.481.54+0-0.051.32+0.05+0.050.33-0.05+0
95 3 0 3.62+0+0.011.81-0.06-0.055.44-0.5-0.380.55-0.5+0
95 2.2 0+0.06+01.371.43-0.05-0.061.27-0.01-0.060.28
95 3 0+0+0.06 3.351.71-0.01+0.054.72-0.82-0.770.55

OS Ver.6 7 8 9 10
3.11 2.2 8.56+0.01-0.162.09+0+0.050.77-0.05-0.054.563.7 3+0.01+0
95 2.2 8.85-0.18-0.292.140.61+0.05+0.044.55+0.01+0.01 3.74+1.15+1.09
95 3 11.92+0.33+0.063.91.21-0.06-0.0118.9+0.44-1.16 8.24+0.55+0.99
95 2.2 9.72+0+0.281.21-0.07+00.38+0.06+03.35-0.06+0 2.42
95 3 10.05-0.11-0.441.92+0-0.050.93+0.06-011.2+0+1.715.71+0.77+1.45
95 2.2 8.9+0.22+0.221.10.38-0.05+03.02-0.05+0 2.2
95 3 8.62+0.33+0.111.540.1711.43+0.87+0.924.94+1.27-0.52
95 2.2 8.24-0.17+0.110.990.33-0.05+02.86-0.06-0.06 2.03+0+0.05
95 3 8.08-0.06+0.101.65-0.06+00.94+0.05+0.0110.43+0.7-0.064.89-0.0.33-0.94

1. Timing[3^10000; ]
2. Timing[10000!; ]
3. hil = Table[1/(i + j - 1), {i, 30}, {j, 30}];
Timing[Det[hil]]
4. Timing[ListPlot[Table[Prime[i], {i, 10000}], PlotJoined -> True]]
5. Timing[N[Pi,3500]]
6. First[Timing[Eigenvalues[Table[Random[],{200},{200}]]]]
7. Timing[Factor[x^92259-1];]
8. Timing[Integrate[1/(1-x^3),x]]
9. Timing[ParametricPlot3D[{r*Cos[Cos[r]]*Cos[psi],r*Cos[Cos[r]]*Sin[psi],
r*Sin[Cos[r]]},{r, 0.001, (9*Pi)/2 + 0.001}, {psi, 0, (3*Pi)/2},PlotPoints ->{72, 24}]]
10. bx = 6 Cos[u] (1 + Sin[u]);by = 16 Sin[u];rad = 4 (1 - Cos[u] / 2);
X = If[Pi < u <= 2 pi, bx + rad cos[v + pi], bx + rad cos[u] cos[v]];
Y = If[Pi < u <= 2 pi, by,by + rad sin[u] cos[v]];
Z = rad Sin[v];
Timing[ParametricPlot3D[{X, Y, Z},{u, 0, 2 Pi}, {v, 0, 2 Pi},PlotPoints ->
{48,12},Axes -> False,Boxed -> False,ViewPoint-> {1.4, -2.6, -1.7}];]



Mathematica timing tests by Karl Unterkofler.
(Karl.ma) (MMA 3.0 Benchmark Test)

OS Ver. 1 2 3 4 5 6 7
3.11 2.2 5.17-0.01+07.86-0.01-0.015.16+0.05+0.058.24+0+0.16 2.31-0.01+0.054.78-0.05+0.1121.53-1.21-1.15
95 2.2 5.27-0.05-0.057.915.27-0.05+0.018.57-0.28-0.272. 3+0.01+0.014.89-0.06-0.0620.65+0+0.17
95 3 6.76+0.05+0.05 10.66-0.11-0.069.06+0.06+0.019.56+0.27+0.161.87-0.05-0.0 67.58+0+0.051.7+0-0.05
95 2.2 1.652.47+0-0.061.65+0.06+09.99+0.28-0.0.321.43+0.06-0.052.91+0+0.0112.13+0.01+0.06
95 3 3.3-0.0.06-0.06 4.73+0.16-0.014+0.33+0.016.65+0.22-0.060.99-0.01-0.063.63-0.12-0.170.66-0.16-0.17
95 2.2 1.48+0.01+0.0152.25-0.05-0.051.48+0+0.019.06+.06-0.111.26-0.05+0.012.69-0.06-0.0610.93+0.11+0
95 3 2.91 4.29-0.06-0.013.63+0-0.016.21-0.22-0.280.82+0.06+0.063.24-0.16-0.060.6-0.16-0.16
95 2.2 1.38-0.01-0.012.031.378.46+0.16-0.281.15+0.01+02.42+0-0.0110.1+0.01+0
95 3 2.74-0.05+0.01 3.95+0.01+0.013.4+0.01+0.015.93-0.16-0.160.82+0+0.013.02-0.10-0.100.55-0.17-0.11

1. Timing[N[Sin[1/2], 2500]][[1]]
2. Timing[N[Pi, 10001]][[1]]
3. Timing[10001!][[1]]
4. First[Timing[Eigenvalues[Table[Random[], {200}, {200}]]]]
5. f[x_] := 4*x - 4*x^2;
Timing[Nest[f, 0.6, 5000]][[1]]
6. f[x_] := BesselJ[0, x];
Timing[Nest[f, 0.6, 2500]][[1]]
7. kdv[q_] := D[q, t] - 1/4*D[q, {x, 3}] - (3*q*D[q, x])/2
q3 := (-5*E^((11*t)/8) - 45*E^(2*x) - 18*E^((11*t)/16 + x) +
162*E^((3*t)/2 + 2*x) - 188*E^((13*t)/16 + 3*x) + 162*E^(t/8 + 4*x) -
45*E^((13*t)/8 + 4*x) - 18*E^((15*t)/16 + 5*x) - 5*E^(t/4 + 6*x))/
(8*(-E^((11*t)/16) + 3*E^x - 3*E^((13*t)/16 + 2*x) + E^(t/8 + 3*x))^2)
Timing[Simplify[kdv[q3]]]


GRTensorM Benchmarks

OS Ver. 1 1 12 22
MaxMemoryUsed[]TimeUsed[]Load TimeMaxMemoryUsed[]TimeUsed[]Load Time
3.11 2.21.3113.85-0.28-0.11~51.2816.19+0.4 4+0.39~5
95 2.21.3114.56-0.6-0.56~51.2816.49-0.31+ 0.49~5
95 31.7325.03+0.52+0.17~141.6933.52+0.5 2+0.59~16
95 2.21.319.22-0.38-0.55~31.2810.07-0.02-0.07~3.5
95 31.7310.43+0.08-0.16~5.51.6914.87+0.16+0.10~6
95 2.21.317.97-0.27-0.28~31.289.01-0.11-0.15~3.5
95 31.739.88-0.62-0.54~5.51.6912.86-0.18-0.11~6
95 2.21.317.41-0.16-0.10~31.288.4+0.04-0.05~3.5
95 31.739.32-0.64-0.80~5.51.6912.24-0.11+0.07~6

OS Ver. 3 3 34 44
MaxMemoryUsed[]TimeUsed[]Load TimeMaxMemoryUsed[]TimeUsed[]Load Time
3.11 2.21.57131.99-0.18-0.12~53.34157.24+0 .52+0.36~13
95 2.21.57133.31-0.34-0.17~53.34157.52-0. 22-0.12~14
95 31.99194.49-1.91-1.43~154.73849.67-2. 91-2.21~34
95 2.21.5782.83-0.07-0.11~33.3785.35-0.21-0.99~8
9531.9997.36+0.91+0.29~54.73398.99+5.27+4.88~16
95 2.21.5774.43+0-0.0.21~33.3776.08-0.22-0.11~8
9531.9987.52-0.20+0.03~54.73360.9+16.8+5.68~16
95 2.21.5768.87-0.0.11-0.10~33.3770.81-0.0.12+0.37~8
9531.9981.38-0.04+0.12~54.73342.14-0.02+0.04~16

1. Reduction of the Ricci tensor to zero and calculation and simplification of the Kretschmann scalar (RabcdRabcd) in Kruskal coordinates for the Schwarzschild space-time by way of constraints. (Kruskal.ma)
2. Calculation and simplification of the invariants R,RabRab,RabcdRabcd,R, a,a,Rab;cRab;c in the Reissner-Nordström-de Sitter space-time.(Radrnds.ma)
3. Calculation and simplification of the differential invariant (Rabcd;eRabcd;e) in the Kerr space-time. The invariant vanishes on the stationary limit surface.(Statlim.ma)
4. Calculation and simplification of all Ricci invariants of the mixmaster space-time.(Mixmr.ma)
Note:The difference between the calculation times for Mathematica 2 and Mathematica 3 for case 4 can be traced to the simplification of the last Ricci invariant (S3). Mathematica 3 produces a more fully simplified form of the invariant via Simplify. Prior to Simplify S3 is 0.43 MB and after 0.19 MB in Mathematica 3.0. In Mathematica 2.2 the numbers are 0.37 MB and 0.35 MB.
End Notes:

i)Why the different times?

A given programme solving a given problem under identical conditions does not take different lengths of time. The catch here is that the conditions are never identical. The state of a computer depends primarily on the state of its cache memory, and this is in fact different even between fresh boots of a system. If the CPU times are small (as they are for some of the "standard tests") it is important to estimate the variation in execution times one might expect.

ii)There is even some variation between the same calculation (First[Timing[Eigenvalues[Table[Random[],{200},{200}]]]]) in the two tables, 6 in the first table and 4 in the second. Why?

The state of the kernel is different due to different calculation streams.

iii)Do the Pentium and Pentium Pro not respond the same way to "Eigenvalues"?

Apparently not.

iv)Is that 0.17 in column 8 of the first table a misprint?

It came back three times.