John Kessenich | 54f6e56 | 2013-08-03 00:04:10 +0000 | [diff] [blame] | 1 | An OpenGL and OpenGL ES shader front end and validator. |
John Kessenich | a0af473 | 2012-12-12 21:15:54 +0000 | [diff] [blame] | 2 | |
John Kessenich | 54f6e56 | 2013-08-03 00:04:10 +0000 | [diff] [blame] | 3 | There are two components: |
John Kessenich | a0af473 | 2012-12-12 21:15:54 +0000 | [diff] [blame] | 4 | |
John Kessenich | 54f6e56 | 2013-08-03 00:04:10 +0000 | [diff] [blame] | 5 | 1) A front-end library for programmatic parsing of GLSL/ESSL into an AST. |
John Kessenich | a0af473 | 2012-12-12 21:15:54 +0000 | [diff] [blame] | 6 | |
John Kessenich | 54f6e56 | 2013-08-03 00:04:10 +0000 | [diff] [blame] | 7 | 2) A standalone wrapper, glslangValidator, that can be used as a shader |
| 8 | validation tool. |
John Kessenich | a0af473 | 2012-12-12 21:15:54 +0000 | [diff] [blame] | 9 | |
John Kessenich | 61c2d14 | 2013-10-03 20:23:57 +0000 | [diff] [blame] | 10 | How to add a feature protected by a version/extension/stage/profile: See the |
| 11 | comment in glslang/MachineIndependent/Versions.cpp. |
| 12 | |
John Kessenich | 54f6e56 | 2013-08-03 00:04:10 +0000 | [diff] [blame] | 13 | Things left to do: See Todo.txt |
John Kessenich | a0af473 | 2012-12-12 21:15:54 +0000 | [diff] [blame] | 14 | |
John Kessenich | 61c2d14 | 2013-10-03 20:23:57 +0000 | [diff] [blame] | 15 | Execution of Standalone Wrapper |
| 16 | ------------------------------- |
John Kessenich | a0af473 | 2012-12-12 21:15:54 +0000 | [diff] [blame] | 17 | |
John Kessenich | 54f6e56 | 2013-08-03 00:04:10 +0000 | [diff] [blame] | 18 | There are binaries in the Install/Windows and Install/Linux directories. |
John Kessenich | a0af473 | 2012-12-12 21:15:54 +0000 | [diff] [blame] | 19 | |
John Kessenich | 54f6e56 | 2013-08-03 00:04:10 +0000 | [diff] [blame] | 20 | To use the standalone binary form, execute glslangValidator, and it will print |
| 21 | a usage statement. Basic operation is to give it a file containing a shader, |
| 22 | and it will print out warnings/errors and optionally an AST. |
John Kessenich | a0af473 | 2012-12-12 21:15:54 +0000 | [diff] [blame] | 23 | |
John Kessenich | c027579 | 2013-08-09 17:14:49 +0000 | [diff] [blame] | 24 | The applied stage-specific rules are based on the file extension: |
| 25 | .vert for a vertex shader |
| 26 | .tesc for a tessellation control shader |
| 27 | .tese for a tessellation evaluation shader |
| 28 | .geom for a geometry shader |
| 29 | .frag for a fragment shader |
| 30 | .comp for a compute shader |
John Kessenich | a0af473 | 2012-12-12 21:15:54 +0000 | [diff] [blame] | 31 | |
John Kessenich | 61c2d14 | 2013-10-03 20:23:57 +0000 | [diff] [blame] | 32 | There is also a non-shader extension |
| 33 | .conf for a configuration file of limits, see usage statement for example |
| 34 | |
| 35 | Source: Build and run on Linux |
John Kessenich | a0af473 | 2012-12-12 21:15:54 +0000 | [diff] [blame] | 36 | ------------------------------- |
| 37 | |
John Kessenich | 54f6e56 | 2013-08-03 00:04:10 +0000 | [diff] [blame] | 38 | A simple bash script "BuildLinux.sh" is provided at the root directory |
| 39 | to do the build and run a test cases. You will need a recent version of |
| 40 | bison installed. |
John Kessenich | a0af473 | 2012-12-12 21:15:54 +0000 | [diff] [blame] | 41 | |
| 42 | Once the executable is generated, it needs to be dynamically linked with the |
John Kessenich | 54f6e56 | 2013-08-03 00:04:10 +0000 | [diff] [blame] | 43 | shared object created in lib directory. To achieve that, "cd" to |
John Kessenich | a0af473 | 2012-12-12 21:15:54 +0000 | [diff] [blame] | 44 | StandAlone directory to update the LD_LIBRARY_PATH as follows |
| 45 | |
| 46 | export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:./../glslang/MachineIndependent/lib |
| 47 | |
| 48 | You can also update LD_LIBRARY_PATH in the .cshrc or .bashrc file, depending on |
| 49 | the shell you are using. You will need to give the complete path of "lib" directory |
| 50 | in .cshrc or .bashrc files. |
| 51 | |
John Kessenich | 54f6e56 | 2013-08-03 00:04:10 +0000 | [diff] [blame] | 52 | Source: Build and run on Windows |
| 53 | -------------------------------- |
John Kessenich | a0af473 | 2012-12-12 21:15:54 +0000 | [diff] [blame] | 54 | |
John Kessenich | 54f6e56 | 2013-08-03 00:04:10 +0000 | [diff] [blame] | 55 | Current development is with Visual Studio verion 11 (2012). The solution |
| 56 | file is in the project's root directory Standalone.sln. |
John Kessenich | a0af473 | 2012-12-12 21:15:54 +0000 | [diff] [blame] | 57 | |
John Kessenich | 54f6e56 | 2013-08-03 00:04:10 +0000 | [diff] [blame] | 58 | Building the StandAlone project (the default) will create glslangValidate.exe and |
| 59 | copy it into the Test directory and Install directory. This allows local |
| 60 | test scripts to use either the debug or release version, whichever was |
| 61 | built last. |
John Kessenich | a0af473 | 2012-12-12 21:15:54 +0000 | [diff] [blame] | 62 | |
John Kessenich | 54f6e56 | 2013-08-03 00:04:10 +0000 | [diff] [blame] | 63 | Windows execution and testing is generally done from within a cygwin |
| 64 | shell. |
John Kessenich | a0af473 | 2012-12-12 21:15:54 +0000 | [diff] [blame] | 65 | |
John Kessenich | 54f6e56 | 2013-08-03 00:04:10 +0000 | [diff] [blame] | 66 | Note: Despite appearances, the use of a DLL is currently disabled; it |
| 67 | simply makes a standalone executable from a statically linked library. |
John Kessenich | a0af473 | 2012-12-12 21:15:54 +0000 | [diff] [blame] | 68 | |
John Kessenich | 2d2f316 | 2013-12-10 00:25:14 +0000 | [diff] [blame] | 69 | Programmatic Interfaces |
| 70 | ----------------------- |
John Kessenich | 4586dbd | 2013-08-05 15:52:03 +0000 | [diff] [blame] | 71 | |
| 72 | Another piece of software can programmatically translate shaders to an AST |
John Kessenich | 2d2f316 | 2013-12-10 00:25:14 +0000 | [diff] [blame] | 73 | using one of two different interfaces: |
| 74 | - A new C++ class-oriented interface, or |
| 75 | - The original C functional interface |
John Kessenich | 4586dbd | 2013-08-05 15:52:03 +0000 | [diff] [blame] | 76 | |
John Kessenich | 2d2f316 | 2013-12-10 00:25:14 +0000 | [diff] [blame] | 77 | The main() in StandAlone/StandAlone.cpp shows examples using both styles. |
John Kessenich | 4586dbd | 2013-08-05 15:52:03 +0000 | [diff] [blame] | 78 | |
John Kessenich | 2d2f316 | 2013-12-10 00:25:14 +0000 | [diff] [blame] | 79 | C++ Class Interface (new, preferred): |
John Kessenich | 4586dbd | 2013-08-05 15:52:03 +0000 | [diff] [blame] | 80 | |
John Kessenich | 2d2f316 | 2013-12-10 00:25:14 +0000 | [diff] [blame] | 81 | This interface is in roughly the last 1/3 of ShaderLang.h. It is in the |
| 82 | glslang namespace and contains the following. |
John Kessenich | 4586dbd | 2013-08-05 15:52:03 +0000 | [diff] [blame] | 83 | |
John Kessenich | 2d2f316 | 2013-12-10 00:25:14 +0000 | [diff] [blame] | 84 | const char* GetEsslVersionString(); |
| 85 | const char* GetGlslVersionString(); |
| 86 | bool InitializeProcess(); |
| 87 | void FinalizeProcess(); |
| 88 | |
| 89 | class TShader |
| 90 | bool parse(...); |
| 91 | void setStrings(...); |
| 92 | const char* getInfoLog(); |
| 93 | |
| 94 | class TProgram |
| 95 | void addShader(...); |
| 96 | bool link(...); |
| 97 | const char* getInfoLog(); |
| 98 | Reflection queries |
| 99 | |
| 100 | See ShaderLang.h and the usage of it in StandAlone/StandAlone.cpp for more |
| 101 | details. |
| 102 | |
| 103 | C Functional Interface (orginal): |
| 104 | |
| 105 | This interface is in roughly the first 2/3 of ShaderLang.h, and referred to |
| 106 | as the Sh*() interface, as all the entry points start "Sh". |
| 107 | |
| 108 | The Sh*() interface takes a "compiler" call-back object, which it calls after |
| 109 | building call back that is passed the AST and can then execute a backend on it. |
| 110 | |
| 111 | The following is a simplified resulting run-time call stack: |
| 112 | |
| 113 | ShCompile(shader, compiler) -> compiler(AST) -> <back end> |
| 114 | |
| 115 | In practice, ShCompile() takes shader strings, default version, and |
| 116 | warning/error and other options for controling compilation. |
John Kessenich | 4586dbd | 2013-08-05 15:52:03 +0000 | [diff] [blame] | 117 | |
| 118 | Testing |
| 119 | ------- |
| 120 | |
| 121 | "Test" is an active test directory that contains test input and a |
| 122 | subdirectory baseResults that contains the expected results of the |
| 123 | tests. Both the tests and baseResults are under source-code control. |
| 124 | Executing the script ./runtests will generate current results in |
| 125 | the localResults directory and diff them against the baseResults. |
| 126 | When you want to update the tracked test results, they need to be |
| 127 | copied from localResults to baseResults |
| 128 | |
| 129 | There are some tests borrowed from LunarGLASS. If LunarGLASS is |
| 130 | missing, those tests just won't run. |
| 131 | |
| 132 | Basic Internal Operation |
| 133 | ------------------------ |
John Kessenich | a0af473 | 2012-12-12 21:15:54 +0000 | [diff] [blame] | 134 | |
John Kessenich | 2d2f316 | 2013-12-10 00:25:14 +0000 | [diff] [blame] | 135 | - Initial lexical analysis is done by the preprocessor in |
John Kessenich | 54f6e56 | 2013-08-03 00:04:10 +0000 | [diff] [blame] | 136 | MachineIndependent/Preprocessor, and then refined by a GLSL scanner |
John Kessenich | acc55c2 | 2013-08-05 17:09:24 +0000 | [diff] [blame] | 137 | in MachineIndependent/Scan.cpp. There is currently no use of flex. |
John Kessenich | a0af473 | 2012-12-12 21:15:54 +0000 | [diff] [blame] | 138 | |
John Kessenich | acc55c2 | 2013-08-05 17:09:24 +0000 | [diff] [blame] | 139 | - Code is parsed using bison on MachineIndependent/glslang.y with the |
| 140 | aid of a symbol table and an AST. The symbol table is not passed on to |
John Kessenich | a0af473 | 2012-12-12 21:15:54 +0000 | [diff] [blame] | 141 | the back-end; the intermediate representation stands on its own. |
John Kessenich | acc55c2 | 2013-08-05 17:09:24 +0000 | [diff] [blame] | 142 | The tree is built by the grammar productions, many of which are |
| 143 | offloaded into ParseHelper.cpp, and by Intermediate.cpp. |
John Kessenich | a0af473 | 2012-12-12 21:15:54 +0000 | [diff] [blame] | 144 | |
John Kessenich | 54f6e56 | 2013-08-03 00:04:10 +0000 | [diff] [blame] | 145 | - The intermediate representation is very high-level, and represented |
John Kessenich | a0af473 | 2012-12-12 21:15:54 +0000 | [diff] [blame] | 146 | as an in-memory tree. This serves to lose no information from the |
| 147 | original program, and to have efficient transfer of the result from |
John Kessenich | 54f6e56 | 2013-08-03 00:04:10 +0000 | [diff] [blame] | 148 | parsing to the back-end. In the AST, constants are propogated and |
John Kessenich | acc55c2 | 2013-08-05 17:09:24 +0000 | [diff] [blame] | 149 | folded, and a very small amount of dead code is eliminated. |
John Kessenich | a0af473 | 2012-12-12 21:15:54 +0000 | [diff] [blame] | 150 | |
John Kessenich | acc55c2 | 2013-08-05 17:09:24 +0000 | [diff] [blame] | 151 | To aid linking and reflection, the last top-level branch in the AST |
| 152 | lists all global symbols. |
John Kessenich | 54f6e56 | 2013-08-03 00:04:10 +0000 | [diff] [blame] | 153 | |
| 154 | - The primary algorithm of the back-end compiler is to traverse the |
John Kessenich | a0af473 | 2012-12-12 21:15:54 +0000 | [diff] [blame] | 155 | tree (high-level intermediate representation), and create an internal |
John Kessenich | 54f6e56 | 2013-08-03 00:04:10 +0000 | [diff] [blame] | 156 | object code representation. There is an example of how to do this |
John Kessenich | acc55c2 | 2013-08-05 17:09:24 +0000 | [diff] [blame] | 157 | in MachineIndependent/intermOut.cpp. |
John Kessenich | a0af473 | 2012-12-12 21:15:54 +0000 | [diff] [blame] | 158 | |
John Kessenich | 54f6e56 | 2013-08-03 00:04:10 +0000 | [diff] [blame] | 159 | - Reduction of the tree to a linear byte-code style low-level intermediate |
John Kessenich | a0af473 | 2012-12-12 21:15:54 +0000 | [diff] [blame] | 160 | representation is likely a good way to generate fully optimized code. |
John Kessenich | acc55c2 | 2013-08-05 17:09:24 +0000 | [diff] [blame] | 161 | |
John Kessenich | 4c70685 | 2013-10-11 16:28:43 +0000 | [diff] [blame] | 162 | - There is currently some dead old-style linker-type code still lying around. |
| 163 | |
| 164 | - Memory pool: parsing uses types derived from C++ std types, using a |
| 165 | custom allocator that puts them in a memory pool. This makes allocation |
| 166 | of individual container/contents just few cycles and deallocation free. |
| 167 | This pool is popped after the AST is made and processed. |
| 168 | |
| 169 | The use is simple: if you are going to call "new", there are three cases: |
| 170 | |
| 171 | - the object comes from the pool (its base class has the macro |
| 172 | POOL_ALLOCATOR_NEW_DELETE in it) and you do not have to call delete |
| 173 | |
| 174 | - it is a TString, in which case call NewPoolTString(), which gets |
| 175 | it from the pool, and there is no corresponding delete |
| 176 | |
| 177 | - the object does not come from the pool, and you have to do normal |
| 178 | C++ memory management of what you 'new' |