David Tolnay | 6eacaff | 2016-10-23 15:20:23 -0700 | [diff] [blame] | 1 | Nom parser for Rust source code |
| 2 | =============================== |
David Tolnay | 35161ff | 2016-09-03 11:33:15 -0700 | [diff] [blame] | 3 | |
David Tolnay | ac9953b | 2016-09-07 08:37:12 -0700 | [diff] [blame] | 4 | [](https://travis-ci.org/dtolnay/syn) |
| 5 | [](https://crates.io/crates/syn) |
David Tolnay | 50e6220 | 2016-09-12 09:49:49 -0700 | [diff] [blame] | 6 | [](https://dtolnay.github.io/syn/syn/) |
David Tolnay | ac9953b | 2016-09-07 08:37:12 -0700 | [diff] [blame] | 7 | |
David Tolnay | 105f2fd | 2017-02-09 09:15:26 -0800 | [diff] [blame^] | 8 | Parse Rust source code without a Syntex dependency, intended for use with |
David Tolnay | 35161ff | 2016-09-03 11:33:15 -0700 | [diff] [blame] | 9 | [Macros 1.1](https://github.com/rust-lang/rfcs/blob/master/text/1681-macros-1.1.md). |
| 10 | |
David Tolnay | f939f35 | 2016-09-11 18:00:09 -0700 | [diff] [blame] | 11 | Designed for fast compile time. |
| 12 | |
David Tolnay | 4415eea | 2017-01-23 18:04:27 -0800 | [diff] [blame] | 13 | - Compile time for `syn` (from scratch including all dependencies): **6 seconds** |
David Tolnay | f939f35 | 2016-09-11 18:00:09 -0700 | [diff] [blame] | 14 | - Compile time for the `syntex`/`quasi`/`aster` stack: **60+ seconds** |
| 15 | |
David Tolnay | fb9f704 | 2016-12-22 12:31:39 -0500 | [diff] [blame] | 16 | If you get stuck with Macros 1.1 I am happy to provide help even if the issue is |
| 17 | not related to syn. Please file a ticket in this repo. |
| 18 | |
David Tolnay | ed8e23e | 2016-09-27 08:18:22 -0700 | [diff] [blame] | 19 | ## Usage with Macros 1.1 |
David Tolnay | 6c9f5b6 | 2016-09-13 15:19:22 -0700 | [diff] [blame] | 20 | |
David Tolnay | 69b538e | 2016-09-23 19:59:48 -0700 | [diff] [blame] | 21 | ```toml |
| 22 | [dependencies] |
David Tolnay | df8fa31 | 2017-01-23 18:22:29 -0800 | [diff] [blame] | 23 | syn = "0.11" |
David Tolnay | f8f4307 | 2016-10-08 14:58:05 -0700 | [diff] [blame] | 24 | quote = "0.3" |
David Tolnay | 69b538e | 2016-09-23 19:59:48 -0700 | [diff] [blame] | 25 | |
| 26 | [lib] |
David Tolnay | 45cc849 | 2016-10-08 20:52:03 -0700 | [diff] [blame] | 27 | proc-macro = true |
David Tolnay | 69b538e | 2016-09-23 19:59:48 -0700 | [diff] [blame] | 28 | ``` |
| 29 | |
White-Oak | 82d0db7 | 2016-09-13 21:45:58 +0300 | [diff] [blame] | 30 | ```rust |
David Tolnay | 45cc849 | 2016-10-08 20:52:03 -0700 | [diff] [blame] | 31 | extern crate proc_macro; |
| 32 | use proc_macro::TokenStream; |
David Tolnay | b4c6326 | 2016-09-23 20:03:06 -0700 | [diff] [blame] | 33 | |
| 34 | extern crate syn; |
David Tolnay | 6c9f5b6 | 2016-09-13 15:19:22 -0700 | [diff] [blame] | 35 | |
White-Oak | 82d0db7 | 2016-09-13 21:45:58 +0300 | [diff] [blame] | 36 | #[macro_use] |
| 37 | extern crate quote; |
White-Oak | 82d0db7 | 2016-09-13 21:45:58 +0300 | [diff] [blame] | 38 | |
David Tolnay | 45cc849 | 2016-10-08 20:52:03 -0700 | [diff] [blame] | 39 | #[proc_macro_derive(MyMacro)] |
David Tolnay | b4c6326 | 2016-09-23 20:03:06 -0700 | [diff] [blame] | 40 | pub fn my_macro(input: TokenStream) -> TokenStream { |
White-Oak | 82d0db7 | 2016-09-13 21:45:58 +0300 | [diff] [blame] | 41 | let source = input.to_string(); |
| 42 | |
David Tolnay | d12e409 | 2016-12-27 22:58:43 -0500 | [diff] [blame] | 43 | // Parse the string representation into a syntax tree |
David Tolnay | 0e83740 | 2016-12-22 17:25:55 -0500 | [diff] [blame] | 44 | let ast = syn::parse_derive_input(&source).unwrap(); |
White-Oak | 82d0db7 | 2016-09-13 21:45:58 +0300 | [diff] [blame] | 45 | |
David Tolnay | 6c9f5b6 | 2016-09-13 15:19:22 -0700 | [diff] [blame] | 46 | // Build the output, possibly using quasi-quotation |
| 47 | let expanded = quote! { |
| 48 | // ... |
| 49 | }; |
| 50 | |
David Tolnay | b988b6d | 2016-10-05 00:12:37 -0700 | [diff] [blame] | 51 | // Parse back to a token stream and return it |
David Tolnay | 62463ba | 2016-11-24 13:06:01 -0800 | [diff] [blame] | 52 | expanded.parse().unwrap() |
White-Oak | 82d0db7 | 2016-09-13 21:45:58 +0300 | [diff] [blame] | 53 | } |
| 54 | ``` |
David Tolnay | 6c9f5b6 | 2016-09-13 15:19:22 -0700 | [diff] [blame] | 55 | |
David Tolnay | b988b6d | 2016-10-05 00:12:37 -0700 | [diff] [blame] | 56 | ## Complete example |
| 57 | |
| 58 | Suppose we have the following simple trait which returns the number of fields in |
| 59 | a struct: |
| 60 | |
| 61 | ```rust |
| 62 | trait NumFields { |
| 63 | fn num_fields() -> usize; |
| 64 | } |
| 65 | ``` |
| 66 | |
| 67 | A complete Macros 1.1 implementation of `#[derive(NumFields)]` based on `syn` |
| 68 | and [`quote`](https://github.com/dtolnay/quote) looks like this: |
| 69 | |
| 70 | ```rust |
David Tolnay | 45cc849 | 2016-10-08 20:52:03 -0700 | [diff] [blame] | 71 | extern crate proc_macro; |
| 72 | use proc_macro::TokenStream; |
David Tolnay | b988b6d | 2016-10-05 00:12:37 -0700 | [diff] [blame] | 73 | |
| 74 | extern crate syn; |
| 75 | |
| 76 | #[macro_use] |
| 77 | extern crate quote; |
| 78 | |
David Tolnay | 45cc849 | 2016-10-08 20:52:03 -0700 | [diff] [blame] | 79 | #[proc_macro_derive(NumFields)] |
David Tolnay | b988b6d | 2016-10-05 00:12:37 -0700 | [diff] [blame] | 80 | pub fn num_fields(input: TokenStream) -> TokenStream { |
| 81 | let source = input.to_string(); |
| 82 | |
David Tolnay | d12e409 | 2016-12-27 22:58:43 -0500 | [diff] [blame] | 83 | // Parse the string representation into a syntax tree |
David Tolnay | 0e83740 | 2016-12-22 17:25:55 -0500 | [diff] [blame] | 84 | let ast = syn::parse_derive_input(&source).unwrap(); |
David Tolnay | b988b6d | 2016-10-05 00:12:37 -0700 | [diff] [blame] | 85 | |
| 86 | // Build the output |
David Tolnay | ed7a508 | 2016-10-30 20:06:29 -0700 | [diff] [blame] | 87 | let expanded = expand_num_fields(&ast); |
David Tolnay | b988b6d | 2016-10-05 00:12:37 -0700 | [diff] [blame] | 88 | |
David Tolnay | 7c509bb | 2016-11-19 13:00:12 -0800 | [diff] [blame] | 89 | // Return the generated impl as a TokenStream |
David Tolnay | 62463ba | 2016-11-24 13:06:01 -0800 | [diff] [blame] | 90 | expanded.parse().unwrap() |
David Tolnay | b988b6d | 2016-10-05 00:12:37 -0700 | [diff] [blame] | 91 | } |
| 92 | |
David Tolnay | 0e83740 | 2016-12-22 17:25:55 -0500 | [diff] [blame] | 93 | fn expand_num_fields(ast: &syn::DeriveInput) -> quote::Tokens { |
David Tolnay | b988b6d | 2016-10-05 00:12:37 -0700 | [diff] [blame] | 94 | let n = match ast.body { |
| 95 | syn::Body::Struct(ref data) => data.fields().len(), |
| 96 | syn::Body::Enum(_) => panic!("#[derive(NumFields)] can only be used with structs"), |
| 97 | }; |
| 98 | |
| 99 | // Used in the quasi-quotation below as `#name` |
| 100 | let name = &ast.ident; |
| 101 | |
| 102 | // Helper is provided for handling complex generic types correctly and effortlessly |
| 103 | let (impl_generics, ty_generics, where_clause) = ast.generics.split_for_impl(); |
| 104 | |
| 105 | quote! { |
David Tolnay | b988b6d | 2016-10-05 00:12:37 -0700 | [diff] [blame] | 106 | // The generated impl |
| 107 | impl #impl_generics ::mycrate::NumFields for #name #ty_generics #where_clause { |
| 108 | fn num_fields() -> usize { |
| 109 | #n |
| 110 | } |
| 111 | } |
| 112 | } |
| 113 | } |
| 114 | ``` |
| 115 | |
David Tolnay | 736829a | 2016-12-22 15:55:53 -0500 | [diff] [blame] | 116 | ## Testing |
| 117 | |
| 118 | Macros 1.1 has a restriction that your proc-macro crate must export nothing but |
| 119 | `proc_macro_derive` functions, and also `proc_macro_derive` procedural macros |
| 120 | cannot be used from the same crate in which they are defined. These restrictions |
| 121 | may be lifted in the future but for now they make writing tests a bit trickier |
| 122 | than for other types of code. |
| 123 | |
| 124 | In particular, you will not be able to write test functions like `#[test] fn |
| 125 | it_works() { ... }` in line with your code. Instead, either put tests in a |
| 126 | [`tests` directory](https://doc.rust-lang.org/book/testing.html#the-tests-directory) |
| 127 | or in a separate crate entirely. |
| 128 | |
| 129 | Additionally, if your procedural macro implements a particular trait, that trait |
| 130 | must be defined in a separate crate from the procedural macro. |
| 131 | |
| 132 | As a concrete example, suppose your procedural macro crate is called `my_derive` |
| 133 | and it implements a trait called `my_crate::MyTrait`. Your unit tests for the |
| 134 | procedural macro can go in `my_derive/tests/test.rs` or into a separate crate |
| 135 | `my_tests/tests/test.rs`. Either way the test would look something like this: |
| 136 | |
| 137 | ```rust |
David Tolnay | 736829a | 2016-12-22 15:55:53 -0500 | [diff] [blame] | 138 | #[macro_use] |
| 139 | extern crate my_derive; |
| 140 | |
| 141 | extern crate my_crate; |
| 142 | use my_crate::MyTrait; |
| 143 | |
| 144 | #[test] |
| 145 | fn it_works() { |
| 146 | #[derive(MyTrait)] |
| 147 | struct S { /* ... */ } |
| 148 | |
| 149 | /* test the thing */ |
| 150 | } |
| 151 | ``` |
| 152 | |
David Tolnay | 941c092 | 2016-12-22 16:06:27 -0500 | [diff] [blame] | 153 | ## Debugging |
| 154 | |
| 155 | When developing a procedural macro it can be helpful to look at what the |
| 156 | generated code looks like. Use `cargo rustc -- -Zunstable-options |
| 157 | --pretty=expanded` or the |
| 158 | [`cargo expand`](https://github.com/dtolnay/cargo-expand) subcommand. |
| 159 | |
| 160 | To show the expanded code for some crate that uses your procedural macro, run |
| 161 | `cargo expand` from that crate. To show the expanded code for one of your own |
| 162 | test cases, run `cargo expand --test the_test_case` where the last argument is |
| 163 | the name of the test file without the `.rs` extension. |
| 164 | |
David Tolnay | 3bfbd54 | 2017-01-16 14:57:53 -0800 | [diff] [blame] | 165 | This write-up by Brandon W Maister discusses debugging in more detail: |
| 166 | [Debugging Rust's new Custom Derive |
| 167 | system](https://quodlibetor.github.io/posts/debugging-rusts-new-custom-derive-system/). |
| 168 | |
David Tolnay | 686f504 | 2016-10-30 19:24:51 -0700 | [diff] [blame] | 169 | ## Optional features |
| 170 | |
| 171 | Syn puts a lot of functionality behind optional features in order to optimize |
| 172 | compile time for the most common use cases. These are the available features and |
| 173 | their effect on compile time. Dependencies are included in the compile times. |
| 174 | |
| 175 | Features | Compile time | Functionality |
| 176 | --- | --- | --- |
David Tolnay | 4415eea | 2017-01-23 18:04:27 -0800 | [diff] [blame] | 177 | *(none)* | 3 sec | The data structures representing the AST of Rust structs, enums, and types. |
| 178 | parsing | 6 sec | Parsing Rust source code containing structs and enums into an AST. |
| 179 | printing | 4 sec | Printing an AST of structs and enums as Rust source code. |
| 180 | **parsing, printing** | **6 sec** | **This is the default.** Parsing and printing of Rust structs and enums. This is typically what you want for implementing Macros 1.1 custom derives. |
| 181 | full | 4 sec | The data structures representing the full AST of all possible Rust code. |
| 182 | full, parsing | 9 sec | Parsing any valid Rust source code to an AST. |
| 183 | full, printing | 6 sec | Turning an AST into Rust source code. |
| 184 | full, parsing, printing | 11 sec | Parsing and printing any Rust syntax. |
David Tolnay | ed7a508 | 2016-10-30 20:06:29 -0700 | [diff] [blame] | 185 | |
David Tolnay | 35161ff | 2016-09-03 11:33:15 -0700 | [diff] [blame] | 186 | ## License |
| 187 | |
| 188 | Licensed under either of |
| 189 | |
| 190 | * Apache License, Version 2.0 ([LICENSE-APACHE](LICENSE-APACHE) or http://www.apache.org/licenses/LICENSE-2.0) |
| 191 | * MIT license ([LICENSE-MIT](LICENSE-MIT) or http://opensource.org/licenses/MIT) |
| 192 | |
| 193 | at your option. |
| 194 | |
| 195 | ### Contribution |
| 196 | |
| 197 | Unless you explicitly state otherwise, any contribution intentionally submitted |
| 198 | for inclusion in this crate by you, as defined in the Apache-2.0 license, shall |
| 199 | be dual licensed as above, without any additional terms or conditions. |