TOBA

Benchmark Results for Toba

Note: these are very dated, but they're still interesting.


We have run several benchmarks to test the performance of Toba-generated applications. We have used both the UCSD Java Microbenchmarks and several applications to compare our performance against existing systems. We actually use a modified version of the UCSD benchmarks that includes additional tests for more thorough testing of synchronization and exception handling. Benchmarks include:

Benchmark Environment

We ran each benchmark on a Sun SparcStation 20 with 2 processors running Solaris 2.5.1. Where appropriate, we compiled all programs and benchmarks with the Sun's commercial SunPro cc. We used full optimization (-xO4 -xcg92) for all compilation. We compared code generated using Toba-generated code to using Sun's Interpreter (JDK Version 1.0.2), the Kaffe Interpreter, and the Guava Java JIT compiler (Version 1.0 Beta 1).

Microbenchmarks (UCSD + others)

Description of Benchmarks

Results

Benchmark JDK 1.0.2 Kaffe Guava Toba
Arithmetic (sec) (sec) (sec) (sec)
Add ints 3.236 18.765 0.150 0.117
Mult Ints 3.496 17.803 0.216 0.185
Add Doubles 4.123 23.729 0.508 0.274
Mult Doubles 4.156 23.105 0.488 0.279
Class Access
Read integer instance variable 2.796 15.294 0.172 0.102
Call to method in same object 3.119 17.884 0.354 0.235
Call to method in another object 3.741 22.400 0.410 0.268
invoke interface method 4.130 24.726 1.397 1.405
Throw Exceptions
local 5.026 19.424 17.634 1.325
non-local 7.315 34.722 33.268 6.616
Catch exception past 1 intervening catch clause 11.516 37.006 39.609 8.079
Catch exception 10 calls away 41.245 392.769 124.182 8.177
Synchronization - Single-threaded program
synchronized blocks 16.088 12.400 6.349 1.683
synchronized methods 15.914 19.852 6.587 2.546
Synchronization - Multi-threaded program
synchronized blocks 16.135 11.867 6.259 3.519
synchronized methods 16.114 20.092 6.518 4.596
Miscellaneous Benchmarks
Empty Loop 1.115 4.547 0.071 0.033
Assign to Integer Array 2.781 12.357 0.294 0.186
Runtime.gc() 0.022 0.025 abort 0.403
3 Threads doing 10,000 yields 2.724 3.284 1.952 0.485
100,000 1-byte writes 4.671 20.039 3.898 3.125
1 100,000 byte write 0.009 0.008 0.009 0.006

Discussion of Results

In virtually all cases, programs compiled with Toba run much faster than the other systems. In particular,

Application Benchmarks

Description of Benchmarks

Results

Benchmark JDK 1.0.2 (sec) Guava (sec) Toba (sec)
javac 45.01 17.85 12.8
espresso 24.9 abort 4.9
Toba 59.8 36.8 20.1
JavaLex 176.3 80.8 38.8
JavaCup 8.3 6.2 2.9
JHLZip 50.7 abort 5.8
JHLUnzip 52.2 abort 5.9

Discussion

Applications compiled using Toba run significantly faster than when using the JDK. Programs generated using Toba are also faster than running with the Guava JIT system, and Toba programs run to completion in several applications where the Guava system fails.
Toba: A Java-to-C Translator
Sumatra Project
Department of Computer Science
Last modified: Thu Nov 7 07:47:12 MST 1996