| |
| <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> |
| <html><head><title>LLVM Test Results for $DateString</title></head> |
| |
| <body bgcolor=white> |
| <center><font size=+3 face=Verdana><b>LLVM Test Results for $DateString</b></font></center> |
| <hr height=1> |
| |
| <table width=100%> |
| <tr><td valign=top align=center> |
| |
| <table border="0" cellspacing="0" cellpadding="2"><tr><td bgcolor="#000000"> |
| <table border="0" cellpadding="5" cellspacing="0"><tr><td bgcolor="#DDAA77"> |
| <font size=+1><b>Sections:</b></font><br> |
| </td></tr><tr><td bgcolor="#FFCC99" align=center> |
| <a href="#Overview">Overview</a><br> |
| <a href="#Changes">Changes</a><br> |
| <a href="#Trends">Trends</a><br> |
| <a href="#Programs">Programs</a><br> |
| <a href="#Regression">Regression</a><br> |
| </td></tr></table></td></tr></table> |
| |
| <p> |
| <table border="0" cellspacing="0" cellpadding="2"><tr><td bgcolor="#000000"> |
| <table border="0" cellpadding="5" cellspacing="0"><tr><td bgcolor="#DDAA77" |
| <font size=+1><b>Previous:</b></font><br> |
| </td></tr><tr><td bgcolor="#FFCC99"> |
| $PrevDaysList |
| </td></tr></table></td></tr></table> |
| <p> |
| |
| <font size=+1><b>Back to:</b></font><br> |
| <a href="..">Test Results</a><br> |
| <a href="../..">LLVM Page</a><p> |
| |
| </td><td valign=top> |
| |
| <center> |
| <table border="0" cellspacing="0" cellpadding="2"><tr><td bgcolor="#000000"> |
| <table border="0" cellpadding="10" cellspacing="0"><tr><td bgcolor="#DDAA77" |
| <font size=+2 face=Verdana><b><a name="Overview">Today's Test Results Overview</font></b> |
| </td></tr></table></td></tr></table></center><p> |
| |
| <!-- Running LOC graph --> |
| <table align=right> |
| <tr><td> |
| <a href="running_loc_large.png" |
| ><img border=0 width=480 height=360 src="running_loc.png"></a> |
| </td></tr> |
| <tr><td align=center>Lines Of Code over Time<br> |
| <font size=-1><a href="running_loc_large.png">Click for larger view</a></font> |
| </td></tr> |
| </table> |
| |
| <h2>CVS Tree Overview:</h2> |
| <ul> |
| <li><a href="$DATE-CVS-Log.txt">CVS Checkout Log</a> |
| <ul> |
| <b>$NumDirsInCVS</b> dirs, <b>$NumFilesInCVS</b> files, <b>$LOC</b> |
| lines of code, checked out in <b>$CVSCheckoutTime</b> seconds<br></ul> |
| <li><a href="$DATE-Build-Log.txt">Compilation Log</a> |
| <ul> |
| $BuildError |
| Time to configure CVS tree: <b>$ConfigTime</b> seconds |
| (<b>$ConfigWallTime</b> seconds wall time)<br> |
| Time to build CVS tree: <b>$BuildTime</b> seconds |
| (<b>$BuildWallTime</b> seconds wall time)<br> |
| Number of object files compiled: <b>$NumObjects</b><br> |
| Number of libraries linked: <b>$NumLibraries</b><br> |
| Number of executables linked:<b> $NumExecutables</b><br> |
| </ul> |
| </ul> |
| |
| <h2>Warnings during the build:</h2> |
| <ul>$WarningsList |
| </ul><p> |
| |
| <br><br><center> |
| <table border="0" cellspacing="0" cellpadding="2"><tr><td bgcolor="#000000"> |
| <table border="0" cellpadding="10" cellspacing="0"><tr><td bgcolor="#DDAA77" |
| <font size=+2 face=Verdana><b><a name="Changes">Changes from Yesterday</font></b> |
| </td></tr></table></td></tr></table></center><p> |
| |
| <h2>Changes to CVS:</h2> |
| <ul> |
| <li>Users who committed to CVS: <b>$UserCommitList</b> |
| <li>Users who updated from CVS: <b>$UserUpdateList</b> |
| <li>Added Files: $AddedFilesList |
| <li>Modified Files: $ModifiedFilesList |
| <li>Removed Files: $RemovedFilesList |
| </ul><p> |
| |
| <h2>Changes to Warnings:</h2> |
| <ul> |
| <li>Warnings Added: $WarningsAdded |
| <li>Warnings Removed: $WarningsRemoved |
| </ul><p> |
| |
| <h2>Changes in the test suite:</h2> |
| <ul> |
| <li>New Tests: $TestsAdded |
| <li>Removed Tests: $TestsRemoved |
| <li>Newly passing tests: $TestsFixed |
| <li>Newly failing tests: $TestsBroken |
| </ul> |
| |
| <br><br><center> |
| <table border="0" cellspacing="0" cellpadding="2"><tr><td bgcolor="#000000"> |
| <table border="0" cellpadding="10" cellspacing="0"><tr><td bgcolor="#DDAA77" |
| <font size=+2 face=Verdana><b><a name="Trends">Changes over Time</font></b> |
| </td></tr></table></td></tr></table></center><p> |
| |
| |
| Here are some charts showing how the LLVM optimizer and code generators are |
| changing over time. For now we use the Olden benchmark suite to measure this, |
| but eventually we will switch to using SPEC CPU2000. All programs are run with |
| "LARGE_PROBLEM_SIZE" enabled. Click on any of the charts to get a larger |
| version.<p> |
| |
| <h2>Compilation Measurements:</h2> |
| |
| <table border="0" align=center> |
| <tr> |
| <td width=50% align=center> |
| <a href="running_Olden_bytecode_large.png"><img width=480 height=360 border=0 src="running_Olden_bytecode.png"></a><br> |
| Size of LLVM bytecode files |
| </td> |
| <td width=50% align=center> |
| <a href="running_Olden_machcode_large.png"><img width=480 height=360 border=0 src="running_Olden_machcode.png"></a><br> |
| Size of native machine code for each program (generated by the JIT) |
| </td></tr> |
| <tr> |
| <td align=center> |
| <a href="running_Olden_opt_time_large.png"><img width=480 height=360 border=0 src="running_Olden_opt_time.png"></a><br> |
| Time to run the LLVM optimizer on each program |
| </td> |
| <td></td></tr> |
| </table> |
| |
| <h2>Program Execution Measurements:</h2> |
| |
| <table border="0" align=center> |
| <tr> |
| <td width=50% align=center> |
| <a href="running_Olden_cbe_time_large.png"><img width=480 height=360 border=0 src="running_Olden_cbe_time.png"></a><br> |
| Execution time for CBE generated executable |
| </td> |
| <td width=50% align=center> |
| <a href="running_Olden_llc_time_large.png"><img width=480 height=360 border=0 src="running_Olden_llc_time.png"></a><br> |
| Execution time for the LLC generated executable |
| </td></tr> |
| |
| <tr> |
| <td align=center> |
| <a href="running_Olden_jit_time_large.png"><img width=480 height=360 border=0 src="running_Olden_jit_time.png"></a><br> |
| Execution time for program in the JIT |
| </td> |
| <td></td></tr> |
| </table> |
| |
| |
| |
| |
| <br><br><center> |
| <table border="0" cellspacing="0" cellpadding="2"><tr><td bgcolor="#000000"> |
| <table border="0" cellpadding="10" cellspacing="0"><tr><td bgcolor="#DDAA77" |
| <font size=+2 face=Verdana><b><a name="Programs">Program Tests</font></b> |
| </td></tr></table></td></tr></table></center><p> |
| |
| This section tests LLVM on a variety of programs in the test suite. This |
| includes benchmark suites like the Olden, McCat, Ptrdist, and SPEC benchmarks as |
| well as a few random programs with test inputs. This section is meant to track |
| how stable LLVM is as a whole. A failure in the execution of any test is marked |
| with an asterisk: `*'. The columns of the tables are:<p> |
| |
| <ol> |
| <li><a name="Program">Program</a> - The name of the program for that row.</li> |
| <li><a name="GCCAS">GCCAS</a> - Time to run LLVM optimizers on the program.</li> |
| <li><a name="Bytecode">Bytecode</a> - The size of the bytecode for the |
| program</li> |
| <li><a name="Instrs">Instrs</a> - The number of LLVM instructions in the |
| compiled bytecode</li> |
| <li><a name="LLC<br>compile">LLC compile</a> - The time taken compile with |
| LLC (the static backend)</li> |
| <li><a name="JIT<br>codegen">JIT codegen</a> - The amount of time spent in the |
| JIT itself, instead of executing the program.</li> |
| <li><a name="Machine<br>code">Machine code</a> - The number of bytes of machine |
| code generated by the JIT.</li> |
| <li><a name="GCC">GCC</a> - The time taken to execute the program when compiled |
| with GCC -O2.</li> |
| <li><a name="CBE">CBE</a> - The time taken to execute the program after |
| compilation through the C backend, compiled with -O2.</li> |
| <li><a name="LLC">LLC</a> - How long does the program generated by the static |
| backend LLC take to execute </li> |
| <li><a name="JIT">JIT</a> - The amount of time spent running the |
| program with the JIT; this includes the code generation phase (listed above) |
| and actually running the program.</li> |
| <li><a name="GCC/LLC">GCC/LLC</a> - The speed-up of the LLC output vs the native |
| GCC output: greater than 1 is a speedup, less than 1 is a slowdown.</li> |
| <li><a name="GCC/CBE">GCC/CBE</a> - The speed-up of the CBE output vs the native |
| GCC output: greater than 1 is a speedup, less than 1 is a slowdown.</li> |
| <li><a name="LLC-LS">LLC-LS</a> - How long does the program generated by the static |
| backend LLC take to execute the program, when compiled with the linear scan |
| register allocator. This is temporary, for tuning.</li> |
| </ol><p> |
| |
| A complete log of testing |
| <a href="$DATE-SingleSource-ProgramTest.txt.gz">SingleSource</a>, |
| <a href="$DATE-MultiSource-ProgramTest.txt.gz">MultiSource</a>, and |
| <a href="$DATE-External-ProgramTest.txt.gz">External</a> programs are |
| available</a> for further analysis. |
| |
| <h2>Programs/External</h2> |
| |
| <center> |
| <table border="0" cellspacing="0" cellpadding="2"><tr><td bgcolor="#000000"> |
| $ExternalProgramsTable |
| </td></tr></table></center> |
| |
| <h2>Programs/MultiSource</h2> |
| |
| <center> |
| <table border="0" cellspacing="0" cellpadding="2"><tr><td bgcolor="#000000"> |
| $MultiSourceProgramsTable |
| </td></tr></table></center> |
| |
| <h2>Programs/SingleSource</h2> |
| |
| <center> |
| <table border="0" cellspacing="0" cellpadding="2"><tr><td bgcolor="#000000"> |
| $SingleSourceProgramsTable |
| </td></tr></table></center> |
| |
| |
| |
| <br><br><center> |
| <table border="0" cellspacing="0" cellpadding="2"><tr><td bgcolor="#000000"> |
| <table border="0" cellpadding="10" cellspacing="0"><tr><td bgcolor="#DDAA77" |
| <font size=+2 face=Verdana><b><a name="Regression">Regression Tests</font></b> |
| </td></tr></table></td></tr></table></center><p> |
| |
| Not yet implemented<p> |
| |
| </td></tr> |
| |
| </html> |