blob: e175677589a94e923efa6a9b63fb0065ac7a5c58 [file] [log] [blame]
David Tolnay55535012018-01-05 16:39:23 -08001// Copyright 2018 Syn Developers
2//
3// Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
4// http://www.apache.org/licenses/LICENSE-2.0> or the MIT license
5// <LICENSE-MIT or http://opensource.org/licenses/MIT>, at your
6// option. This file may not be copied, modified, or distributed
7// except according to those terms.
8
David Tolnay5e84e972018-01-05 17:51:06 -08009//! Syn is a parsing library for parsing a stream of Rust tokens into a syntax
10//! tree of Rust source code.
11//!
12//! Currently this library is geared toward the [custom derive] use case but
13//! contains some APIs that may be useful for Rust procedural macros more
14//! generally.
15//!
16//! [custom derive]: https://github.com/rust-lang/rfcs/blob/master/text/1681-macros-1.1.md
17//!
18//! - **Data structures** — Syn provides a complete syntax tree that can
19//! represent any valid Rust source code. The syntax tree is rooted at
20//! [`syn::File`] which represents a full source file, but there are other
21//! entry points that may be useful to procedural macros including
22//! [`syn::Item`], [`syn::Expr`] and [`syn::Type`].
23//!
24//! - **Custom derives** — Of particular interest to custom derives is
25//! [`syn::DeriveInput`] which is any of the three legal input items to a
26//! derive macro. An example below shows using this type in a library that can
27//! derive implementations of a trait of your own.
28//!
29//! - **Parser combinators** — Parsing in Syn is built on a suite of public
30//! parser combinator macros that you can use for parsing any token-based
31//! syntax you dream up within a `functionlike!(...)` procedural macro. Every
32//! syntax tree node defined by Syn is individually parsable and may be used
33//! as a building block for custom syntaxes, or you may do it all yourself
34//! working from the most primitive tokens.
35//!
36//! - **Location information** — Every token parsed by Syn is associated with a
37//! `Span` that tracks line and column information back to the source of that
38//! token. These spans allow a procedural macro to display detailed error
39//! messages pointing to all the right places in the user's code. There is an
40//! example of this below.
41//!
42//! - **Feature flags** — Functionality is aggressively feature gated so your
43//! procedural macros enable only what they need, and do not pay in compile
44//! time for all the rest.
45//!
46//! [`syn::File`]: struct.File.html
47//! [`syn::Item`]: enum.Item.html
48//! [`syn::Expr`]: enum.Expr.html
49//! [`syn::Type`]: enum.Type.html
50//! [`syn::DeriveInput`]: struct.DeriveInput.html
51//!
52//! *Version requirement: Syn supports any compiler version back to Rust's very
53//! first support for procedural macros in Rust 1.15.0. Some features especially
54//! around error reporting are only available in newer compilers or on the
55//! nightly channel.*
56//!
57//! ## Example of a custom derive
58//!
59//! The canonical custom derive using Syn looks like this. We write an ordinary
60//! Rust function tagged with a `proc_macro_derive` attribute and the name of
61//! the trait we are deriving. Any time that derive appears in the user's code,
62//! the Rust compiler passes their data structure as tokens into our macro. We
63//! get to execute arbitrary Rust code to figure out what to do with those
64//! tokens, then hand some tokens back to the compiler to compile into the
65//! user's crate.
66//!
67//! [`TokenStream`]: https://doc.rust-lang.org/proc_macro/struct.TokenStream.html
68//!
69//! ```toml
70//! [dependencies]
David Tolnay87003d02018-05-20 19:45:13 -070071//! syn = "0.14"
72//! quote = "0.6"
David Tolnay5e84e972018-01-05 17:51:06 -080073//!
74//! [lib]
75//! proc-macro = true
76//! ```
77//!
78//! ```rust
79//! extern crate proc_macro;
80//! extern crate syn;
81//!
82//! #[macro_use]
83//! extern crate quote;
84//!
85//! use proc_macro::TokenStream;
86//! use syn::DeriveInput;
87//!
88//! # const IGNORE_TOKENS: &str = stringify! {
89//! #[proc_macro_derive(MyMacro)]
90//! # };
91//! pub fn my_macro(input: TokenStream) -> TokenStream {
92//! // Parse the input tokens into a syntax tree
93//! let input: DeriveInput = syn::parse(input).unwrap();
94//!
95//! // Build the output, possibly using quasi-quotation
96//! let expanded = quote! {
97//! // ...
98//! };
99//!
100//! // Hand the output tokens back to the compiler
101//! expanded.into()
102//! }
103//! #
104//! # fn main() {}
105//! ```
106//!
107//! The [`heapsize`] example directory shows a complete working Macros 1.1
108//! implementation of a custom derive. It works on any Rust compiler \>=1.15.0.
109//! The example derives a `HeapSize` trait which computes an estimate of the
110//! amount of heap memory owned by a value.
111//!
112//! [`heapsize`]: https://github.com/dtolnay/syn/tree/master/examples/heapsize
113//!
114//! ```rust
115//! pub trait HeapSize {
116//! /// Total number of bytes of heap memory owned by `self`.
117//! fn heap_size_of_children(&self) -> usize;
118//! }
119//! ```
120//!
121//! The custom derive allows users to write `#[derive(HeapSize)]` on data
122//! structures in their program.
123//!
124//! ```rust
125//! # const IGNORE_TOKENS: &str = stringify! {
126//! #[derive(HeapSize)]
127//! # };
128//! struct Demo<'a, T: ?Sized> {
129//! a: Box<T>,
130//! b: u8,
131//! c: &'a str,
132//! d: String,
133//! }
134//! ```
135//!
136//! ## Spans and error reporting
137//!
138//! The [`heapsize2`] example directory is an extension of the `heapsize`
139//! example that demonstrates some of the hygiene and error reporting properties
140//! of Macros 2.0. This example currently requires a nightly Rust compiler
141//! \>=1.24.0-nightly but we are working to stabilize all of the APIs involved.
142//!
143//! [`heapsize2`]: https://github.com/dtolnay/syn/tree/master/examples/heapsize2
144//!
145//! The token-based procedural macro API provides great control over where the
146//! compiler's error messages are displayed in user code. Consider the error the
147//! user sees if one of their field types does not implement `HeapSize`.
148//!
149//! ```rust
150//! # const IGNORE_TOKENS: &str = stringify! {
151//! #[derive(HeapSize)]
152//! # };
153//! struct Broken {
154//! ok: String,
155//! bad: std::thread::Thread,
156//! }
157//! ```
158//!
159//! In the Macros 1.1 string-based procedural macro world, the resulting error
160//! would point unhelpfully to the invocation of the derive macro and not to the
161//! actual problematic field.
162//!
163//! ```text
164//! error[E0599]: no method named `heap_size_of_children` found for type `std::thread::Thread` in the current scope
165//! --> src/main.rs:4:10
166//! |
167//! 4 | #[derive(HeapSize)]
168//! | ^^^^^^^^
169//! ```
170//!
171//! By tracking span information all the way through the expansion of a
172//! procedural macro as shown in the `heapsize2` example, token-based macros in
173//! Syn are able to trigger errors that directly pinpoint the source of the
174//! problem.
175//!
176//! ```text
177//! error[E0277]: the trait bound `std::thread::Thread: HeapSize` is not satisfied
178//! --> src/main.rs:7:5
179//! |
180//! 7 | bad: std::thread::Thread,
David Tolnayefff2ff2018-01-07 11:49:52 -0800181//! | ^^^^^^^^^^^^^^^^^^^^^^^^ the trait `HeapSize` is not implemented for `Thread`
David Tolnay5e84e972018-01-05 17:51:06 -0800182//! ```
183//!
184//! ## Parsing a custom syntax using combinators
185//!
186//! The [`lazy-static`] example directory shows the implementation of a
187//! `functionlike!(...)` procedural macro in which the input tokens are parsed
188//! using [`nom`]-style parser combinators.
189//!
190//! [`lazy-static`]: https://github.com/dtolnay/syn/tree/master/examples/lazy-static
191//! [`nom`]: https://github.com/Geal/nom
192//!
193//! The example reimplements the popular `lazy_static` crate from crates.io as a
194//! procedural macro.
195//!
196//! ```
197//! # macro_rules! lazy_static {
198//! # ($($tt:tt)*) => {}
199//! # }
200//! #
201//! lazy_static! {
202//! static ref USERNAME: Regex = Regex::new("^[a-z0-9_-]{3,16}$").unwrap();
203//! }
204//! ```
205//!
206//! The implementation shows how to trigger custom warnings and error messages
207//! on the macro input.
208//!
209//! ```text
210//! warning: come on, pick a more creative name
211//! --> src/main.rs:10:16
212//! |
213//! 10 | static ref FOO: String = "lazy_static".to_owned();
214//! | ^^^
215//! ```
216//!
217//! ## Debugging
218//!
219//! When developing a procedural macro it can be helpful to look at what the
220//! generated code looks like. Use `cargo rustc -- -Zunstable-options
221//! --pretty=expanded` or the [`cargo expand`] subcommand.
222//!
David Tolnay324db2d2018-01-07 11:51:09 -0800223//! [`cargo expand`]: https://github.com/dtolnay/cargo-expand
David Tolnay5e84e972018-01-05 17:51:06 -0800224//!
225//! To show the expanded code for some crate that uses your procedural macro,
226//! run `cargo expand` from that crate. To show the expanded code for one of
227//! your own test cases, run `cargo expand --test the_test_case` where the last
228//! argument is the name of the test file without the `.rs` extension.
229//!
230//! This write-up by Brandon W Maister discusses debugging in more detail:
231//! [Debugging Rust's new Custom Derive system][debugging].
232//!
233//! [debugging]: https://quodlibetor.github.io/posts/debugging-rusts-new-custom-derive-system/
234//!
235//! ## Optional features
236//!
237//! Syn puts a lot of functionality behind optional features in order to
238//! optimize compile time for the most common use cases. The following features
239//! are available.
240//!
241//! - **`derive`** *(enabled by default)* — Data structures for representing the
242//! possible input to a custom derive, including structs and enums and types.
243//! - **`full`** — Data structures for representing the syntax tree of all valid
244//! Rust source code, including items and expressions.
245//! - **`parsing`** *(enabled by default)* — Ability to parse input tokens into
246//! a syntax tree node of a chosen type.
247//! - **`printing`** *(enabled by default)* — Ability to print a syntax tree
248//! node as tokens of Rust source code.
249//! - **`visit`** — Trait for traversing a syntax tree.
David Tolnay34981cf2018-01-06 16:22:35 -0800250//! - **`visit-mut`** — Trait for traversing and mutating in place a syntax
David Tolnay5e84e972018-01-05 17:51:06 -0800251//! tree.
252//! - **`fold`** — Trait for transforming an owned syntax tree.
253//! - **`clone-impls`** *(enabled by default)* — Clone impls for all syntax tree
254//! types.
255//! - **`extra-traits`** — Debug, Eq, PartialEq, Hash impls for all syntax tree
256//! types.
hcpl4b72a382018-04-04 14:50:24 +0300257//! - **`proc-macro`** *(enabled by default)* — Runtime dependency on the
258//! dynamic library libproc_macro from rustc toolchain.
David Tolnay5e84e972018-01-05 17:51:06 -0800259
David Tolnay4cf7db82018-01-07 15:22:01 -0800260// Syn types in rustdoc of other crates get linked to here.
David Tolnaye6ec9e02018-08-21 21:32:03 -0400261#![doc(html_root_url = "https://docs.rs/syn/0.14.9")]
David Tolnay34071ba2018-05-20 20:00:41 -0700262#![cfg_attr(feature = "cargo-clippy", deny(clippy, clippy_pedantic))]
David Tolnay34071ba2018-05-20 20:00:41 -0700263// Ignored clippy lints.
David Tolnay94d2b792018-04-29 12:26:10 -0700264#![cfg_attr(
265 feature = "cargo-clippy",
266 allow(
David Tolnay0cec3e62018-07-21 09:08:30 -0700267 const_static_lifetime,
David Tolnay24b079d2018-08-27 08:28:10 -0700268 cyclomatic_complexity,
David Tolnay0cec3e62018-07-21 09:08:30 -0700269 doc_markdown,
David Tolnay24b079d2018-08-27 08:28:10 -0700270 eval_order_dependence,
David Tolnay0cec3e62018-07-21 09:08:30 -0700271 large_enum_variant,
272 match_bool,
273 redundant_closure,
274 needless_pass_by_value,
275 redundant_field_names,
276 trivially_copy_pass_by_ref
David Tolnay94d2b792018-04-29 12:26:10 -0700277 )
278)]
David Tolnay34071ba2018-05-20 20:00:41 -0700279// Ignored clippy_pedantic lints.
280#![cfg_attr(
281 feature = "cargo-clippy",
282 allow(
David Tolnay0cec3e62018-07-21 09:08:30 -0700283 cast_possible_truncation,
284 cast_possible_wrap,
David Tolnayb1617752018-08-24 21:16:33 -0400285 empty_enum,
David Tolnay0cec3e62018-07-21 09:08:30 -0700286 if_not_else,
287 indexing_slicing,
288 items_after_statements,
David Tolnay151f92f2018-08-14 22:44:53 -0700289 shadow_unrelated,
David Tolnay0cec3e62018-07-21 09:08:30 -0700290 similar_names,
291 single_match_else,
292 stutter,
293 unseparated_literal_suffix,
294 use_self,
295 used_underscore_binding
David Tolnay34071ba2018-05-20 20:00:41 -0700296 )
297)]
David Tolnayad2836d2017-04-20 10:11:43 -0700298
David Tolnay278f9e32018-08-14 22:41:11 -0700299#[cfg(all(
300 not(all(target_arch = "wasm32", target_os = "unknown")),
301 feature = "proc-macro"
302))]
David Tolnay51382052017-12-27 13:46:21 -0500303extern crate proc_macro;
David Tolnay94d2b792018-04-29 12:26:10 -0700304extern crate proc_macro2;
David Tolnay570695e2017-06-03 16:15:13 -0700305extern crate unicode_xid;
Alex Crichtonccbb45d2017-05-23 10:58:24 -0700306
David Tolnay1cf80912017-12-31 18:35:12 -0500307#[cfg(feature = "printing")]
David Tolnay87d0b442016-09-04 11:52:12 -0700308extern crate quote;
309
Alex Crichton62a0a592017-05-22 13:58:53 -0700310#[macro_use]
311mod macros;
312
David Tolnay852bff72018-08-27 08:24:02 -0700313#[cfg(feature = "parsing")]
314#[macro_use]
315mod group;
316
David Tolnayc5ab8c62017-12-26 16:43:39 -0500317#[macro_use]
David Tolnay32954ef2017-12-26 22:43:16 -0500318pub mod token;
David Tolnayc5ab8c62017-12-26 16:43:39 -0500319
David Tolnay4fb71232018-08-25 23:14:50 -0400320mod ident;
321pub use ident::Ident;
David Tolnaye303b7c2018-05-20 16:46:35 -0700322
David Tolnay3cfd1d32018-01-03 00:22:08 -0800323#[cfg(any(feature = "full", feature = "derive"))]
David Tolnayb79ee962016-09-04 09:39:20 -0700324mod attr;
David Tolnay3cfd1d32018-01-03 00:22:08 -0800325#[cfg(any(feature = "full", feature = "derive"))]
David Tolnayaaadd782018-01-06 22:58:13 -0800326pub use attr::{AttrStyle, Attribute, Meta, MetaList, MetaNameValue, NestedMeta};
David Tolnay35161ff2016-09-03 11:33:15 -0700327
David Tolnay3cfd1d32018-01-03 00:22:08 -0800328#[cfg(any(feature = "full", feature = "derive"))]
David Tolnayf38cdf62016-09-23 19:07:09 -0700329mod data;
David Tolnay3cfd1d32018-01-03 00:22:08 -0800330#[cfg(any(feature = "full", feature = "derive"))]
David Tolnayb57c8492018-05-05 00:32:04 -0700331pub use data::{
332 Field, Fields, FieldsNamed, FieldsUnnamed, Variant, VisCrate, VisPublic, VisRestricted,
333 Visibility,
334};
David Tolnayf38cdf62016-09-23 19:07:09 -0700335
David Tolnay3cfd1d32018-01-03 00:22:08 -0800336#[cfg(any(feature = "full", feature = "derive"))]
David Tolnayf4bbbd92016-09-23 14:41:55 -0700337mod expr;
David Tolnay3cfd1d32018-01-03 00:22:08 -0800338#[cfg(any(feature = "full", feature = "derive"))]
David Tolnayb57c8492018-05-05 00:32:04 -0700339pub use expr::{
David Tolnay02a9c6f2018-08-24 18:58:45 -0400340 Expr, ExprArray, ExprAssign, ExprAssignOp, ExprAsync, ExprBinary, ExprBlock, ExprBox,
341 ExprBreak, ExprCall, ExprCast, ExprClosure, ExprContinue, ExprField, ExprForLoop, ExprGroup,
342 ExprIf, ExprIfLet, ExprInPlace, ExprIndex, ExprLit, ExprLoop, ExprMacro, ExprMatch,
343 ExprMethodCall, ExprParen, ExprPath, ExprRange, ExprReference, ExprRepeat, ExprReturn,
344 ExprStruct, ExprTry, ExprTryBlock, ExprTuple, ExprType, ExprUnary, ExprUnsafe, ExprVerbatim,
345 ExprWhile, ExprWhileLet, ExprYield, Index, Member,
David Tolnayb57c8492018-05-05 00:32:04 -0700346};
Michael Layzell734adb42017-06-07 16:58:31 -0400347
348#[cfg(feature = "full")]
David Tolnayb57c8492018-05-05 00:32:04 -0700349pub use expr::{
350 Arm, Block, FieldPat, FieldValue, GenericMethodArgument, Label, Local, MethodTurbofish, Pat,
351 PatBox, PatIdent, PatLit, PatMacro, PatPath, PatRange, PatRef, PatSlice, PatStruct, PatTuple,
352 PatTupleStruct, PatVerbatim, PatWild, RangeLimits, Stmt,
353};
David Tolnayf4bbbd92016-09-23 14:41:55 -0700354
David Tolnay3cfd1d32018-01-03 00:22:08 -0800355#[cfg(any(feature = "full", feature = "derive"))]
David Tolnayb79ee962016-09-04 09:39:20 -0700356mod generics;
David Tolnay3cfd1d32018-01-03 00:22:08 -0800357#[cfg(any(feature = "full", feature = "derive"))]
David Tolnayb57c8492018-05-05 00:32:04 -0700358pub use generics::{
359 BoundLifetimes, ConstParam, GenericParam, Generics, LifetimeDef, PredicateEq,
360 PredicateLifetime, PredicateType, TraitBound, TraitBoundModifier, TypeParam, TypeParamBound,
361 WhereClause, WherePredicate,
362};
David Tolnay278f9e32018-08-14 22:41:11 -0700363#[cfg(all(
364 any(feature = "full", feature = "derive"),
365 feature = "printing"
366))]
David Tolnayfd6bf5c2017-11-12 09:41:14 -0800367pub use generics::{ImplGenerics, Turbofish, TypeGenerics};
David Tolnay35161ff2016-09-03 11:33:15 -0700368
David Tolnayf38cdf62016-09-23 19:07:09 -0700369#[cfg(feature = "full")]
David Tolnayb79ee962016-09-04 09:39:20 -0700370mod item;
David Tolnayf38cdf62016-09-23 19:07:09 -0700371#[cfg(feature = "full")]
David Tolnayb57c8492018-05-05 00:32:04 -0700372pub use item::{
David Tolnay435c1782018-08-24 16:15:44 -0400373 ArgCaptured, ArgSelf, ArgSelfRef, FnArg, FnDecl, ForeignItem, ForeignItemFn, ForeignItemMacro,
374 ForeignItemStatic, ForeignItemType, ForeignItemVerbatim, ImplItem, ImplItemConst,
David Tolnaybb82ef02018-08-24 20:15:45 -0400375 ImplItemExistential, ImplItemMacro, ImplItemMethod, ImplItemType, ImplItemVerbatim, Item,
376 ItemConst, ItemEnum, ItemExistential, ItemExternCrate, ItemFn, ItemForeignMod, ItemImpl,
377 ItemMacro, ItemMacro2, ItemMod, ItemStatic, ItemStruct, ItemTrait, ItemType, ItemUnion,
David Tolnay3779bb72018-08-26 18:46:07 -0700378 ItemUse, ItemVerbatim, MethodSig, TraitItem, TraitItemConst, TraitItemMacro, TraitItemMethod,
379 TraitItemType, TraitItemVerbatim, UseGlob, UseGroup, UseName, UsePath, UseRename, UseTree,
David Tolnayb57c8492018-05-05 00:32:04 -0700380};
David Tolnay35161ff2016-09-03 11:33:15 -0700381
David Tolnay631cb8c2016-11-10 17:16:41 -0800382#[cfg(feature = "full")]
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700383mod file;
David Tolnay631cb8c2016-11-10 17:16:41 -0800384#[cfg(feature = "full")]
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700385pub use file::File;
David Tolnay631cb8c2016-11-10 17:16:41 -0800386
David Tolnay3cfd1d32018-01-03 00:22:08 -0800387#[cfg(any(feature = "full", feature = "derive"))]
David Tolnay63e3dee2017-06-03 20:13:17 -0700388mod lifetime;
David Tolnay3cfd1d32018-01-03 00:22:08 -0800389#[cfg(any(feature = "full", feature = "derive"))]
David Tolnay63e3dee2017-06-03 20:13:17 -0700390pub use lifetime::Lifetime;
391
David Tolnay3cfd1d32018-01-03 00:22:08 -0800392#[cfg(any(feature = "full", feature = "derive"))]
David Tolnayf4bbbd92016-09-23 14:41:55 -0700393mod lit;
David Tolnay3cfd1d32018-01-03 00:22:08 -0800394#[cfg(any(feature = "full", feature = "derive"))]
David Tolnayb57c8492018-05-05 00:32:04 -0700395pub use lit::{
396 FloatSuffix, IntSuffix, Lit, LitBool, LitByte, LitByteStr, LitChar, LitFloat, LitInt, LitStr,
397 LitVerbatim, StrStyle,
398};
David Tolnayf4bbbd92016-09-23 14:41:55 -0700399
David Tolnay3cfd1d32018-01-03 00:22:08 -0800400#[cfg(any(feature = "full", feature = "derive"))]
David Tolnayf4bbbd92016-09-23 14:41:55 -0700401mod mac;
David Tolnay3cfd1d32018-01-03 00:22:08 -0800402#[cfg(any(feature = "full", feature = "derive"))]
David Tolnayab919512017-12-30 23:31:51 -0500403pub use mac::{Macro, MacroDelimiter};
David Tolnayf4bbbd92016-09-23 14:41:55 -0700404
David Tolnay3cfd1d32018-01-03 00:22:08 -0800405#[cfg(any(feature = "full", feature = "derive"))]
David Tolnay0e837402016-12-22 17:25:55 -0500406mod derive;
David Tolnay3cfd1d32018-01-03 00:22:08 -0800407#[cfg(feature = "derive")]
David Tolnaye3d41b72017-12-31 15:24:00 -0500408pub use derive::{Data, DataEnum, DataStruct, DataUnion, DeriveInput};
David Tolnayf38cdf62016-09-23 19:07:09 -0700409
David Tolnay3cfd1d32018-01-03 00:22:08 -0800410#[cfg(any(feature = "full", feature = "derive"))]
David Tolnay3cb23a92016-10-07 23:02:21 -0700411mod op;
David Tolnay3cfd1d32018-01-03 00:22:08 -0800412#[cfg(any(feature = "full", feature = "derive"))]
David Tolnay3cb23a92016-10-07 23:02:21 -0700413pub use op::{BinOp, UnOp};
414
David Tolnay3cfd1d32018-01-03 00:22:08 -0800415#[cfg(any(feature = "full", feature = "derive"))]
David Tolnayb79ee962016-09-04 09:39:20 -0700416mod ty;
David Tolnay3cfd1d32018-01-03 00:22:08 -0800417#[cfg(any(feature = "full", feature = "derive"))]
David Tolnayb57c8492018-05-05 00:32:04 -0700418pub use ty::{
419 Abi, BareFnArg, BareFnArgName, ReturnType, Type, TypeArray, TypeBareFn, TypeGroup,
420 TypeImplTrait, TypeInfer, TypeMacro, TypeNever, TypeParen, TypePath, TypePtr, TypeReference,
421 TypeSlice, TypeTraitObject, TypeTuple, TypeVerbatim,
422};
David Tolnay056de302018-01-05 14:29:05 -0800423
424#[cfg(any(feature = "full", feature = "derive"))]
425mod path;
David Tolnay278f9e32018-08-14 22:41:11 -0700426#[cfg(all(
427 any(feature = "full", feature = "derive"),
428 feature = "printing"
429))]
David Tolnay94d2b792018-04-29 12:26:10 -0700430pub use path::PathTokens;
David Tolnay056de302018-01-05 14:29:05 -0800431#[cfg(any(feature = "full", feature = "derive"))]
David Tolnayb57c8492018-05-05 00:32:04 -0700432pub use path::{
433 AngleBracketedGenericArguments, Binding, GenericArgument, ParenthesizedGenericArguments, Path,
434 PathArguments, PathSegment, QSelf,
435};
Alex Crichtonccbb45d2017-05-23 10:58:24 -0700436
David Tolnay1b752fb2017-12-26 21:41:39 -0500437#[cfg(feature = "parsing")]
David Tolnaydfc886b2018-01-06 08:03:09 -0800438pub mod buffer;
David Tolnay94d2b792018-04-29 12:26:10 -0700439pub mod punctuated;
David Tolnay1b752fb2017-12-26 21:41:39 -0500440#[cfg(feature = "parsing")]
David Tolnayc5ab8c62017-12-26 16:43:39 -0500441pub mod synom;
David Tolnay3779bb72018-08-26 18:46:07 -0700442#[cfg(all(
443 any(feature = "full", feature = "derive"),
444 feature = "extra-traits"
445))]
David Tolnaye0824032017-12-27 15:25:56 -0500446mod tt;
David Tolnayc5ab8c62017-12-26 16:43:39 -0500447
David Tolnaye83ef5a2018-01-11 15:18:36 -0800448// Not public API except the `parse_quote!` macro.
David Tolnay491680a2018-01-23 00:34:40 -0800449#[cfg(feature = "parsing")]
David Tolnaye83ef5a2018-01-11 15:18:36 -0800450#[doc(hidden)]
451pub mod parse_quote;
452
David Tolnay4d942b42018-01-02 22:14:04 -0800453#[cfg(all(feature = "parsing", feature = "printing"))]
David Tolnayf790b612017-12-31 18:46:57 -0500454pub mod spanned;
455
Nika Layzella6f46c42017-10-26 15:26:16 -0400456mod gen {
David Tolnayded2d682018-01-06 18:53:53 -0800457 /// Syntax tree traversal to walk a shared borrow of a syntax tree.
458 ///
459 /// Each method of the [`Visit`] trait is a hook that can be overridden to
460 /// customize the behavior when visiting the corresponding type of node. By
461 /// default, every method recursively visits the substructure of the input
462 /// by invoking the right visitor method of each of its fields.
463 ///
464 /// [`Visit`]: trait.Visit.html
465 ///
466 /// ```rust
467 /// # use syn::{Attribute, BinOp, Expr, ExprBinary};
468 /// #
469 /// pub trait Visit<'ast> {
470 /// /* ... */
471 ///
472 /// fn visit_expr_binary(&mut self, node: &'ast ExprBinary) {
473 /// for attr in &node.attrs {
474 /// self.visit_attribute(attr);
475 /// }
476 /// self.visit_expr(&*node.left);
477 /// self.visit_bin_op(&node.op);
478 /// self.visit_expr(&*node.right);
479 /// }
480 ///
481 /// /* ... */
482 /// # fn visit_attribute(&mut self, node: &'ast Attribute);
483 /// # fn visit_expr(&mut self, node: &'ast Expr);
484 /// # fn visit_bin_op(&mut self, node: &'ast BinOp);
485 /// }
486 /// ```
David Tolnay461d98e2018-01-07 11:07:19 -0800487 ///
488 /// *This module is available if Syn is built with the `"visit"` feature.*
Nika Layzella6f46c42017-10-26 15:26:16 -0400489 #[cfg(feature = "visit")]
490 pub mod visit;
David Tolnay55337722016-09-11 12:58:56 -0700491
David Tolnayded2d682018-01-06 18:53:53 -0800492 /// Syntax tree traversal to mutate an exclusive borrow of a syntax tree in
493 /// place.
494 ///
495 /// Each method of the [`VisitMut`] trait is a hook that can be overridden
496 /// to customize the behavior when mutating the corresponding type of node.
497 /// By default, every method recursively visits the substructure of the
498 /// input by invoking the right visitor method of each of its fields.
499 ///
500 /// [`VisitMut`]: trait.VisitMut.html
501 ///
502 /// ```rust
503 /// # use syn::{Attribute, BinOp, Expr, ExprBinary};
504 /// #
505 /// pub trait VisitMut {
506 /// /* ... */
507 ///
508 /// fn visit_expr_binary_mut(&mut self, node: &mut ExprBinary) {
509 /// for attr in &mut node.attrs {
510 /// self.visit_attribute_mut(attr);
511 /// }
512 /// self.visit_expr_mut(&mut *node.left);
513 /// self.visit_bin_op_mut(&mut node.op);
514 /// self.visit_expr_mut(&mut *node.right);
515 /// }
516 ///
517 /// /* ... */
518 /// # fn visit_attribute_mut(&mut self, node: &mut Attribute);
519 /// # fn visit_expr_mut(&mut self, node: &mut Expr);
520 /// # fn visit_bin_op_mut(&mut self, node: &mut BinOp);
521 /// }
522 /// ```
David Tolnay461d98e2018-01-07 11:07:19 -0800523 ///
524 /// *This module is available if Syn is built with the `"visit-mut"`
525 /// feature.*
David Tolnay9df02c42018-01-06 13:52:48 -0800526 #[cfg(feature = "visit-mut")]
Nika Layzella6f46c42017-10-26 15:26:16 -0400527 pub mod visit_mut;
Nika Layzell27726662017-10-24 23:16:35 -0400528
David Tolnayded2d682018-01-06 18:53:53 -0800529 /// Syntax tree traversal to transform the nodes of an owned syntax tree.
530 ///
531 /// Each method of the [`Fold`] trait is a hook that can be overridden to
532 /// customize the behavior when transforming the corresponding type of node.
533 /// By default, every method recursively visits the substructure of the
534 /// input by invoking the right visitor method of each of its fields.
535 ///
536 /// [`Fold`]: trait.Fold.html
537 ///
538 /// ```rust
539 /// # use syn::{Attribute, BinOp, Expr, ExprBinary};
540 /// #
541 /// pub trait Fold {
542 /// /* ... */
543 ///
544 /// fn fold_expr_binary(&mut self, node: ExprBinary) -> ExprBinary {
545 /// ExprBinary {
546 /// attrs: node.attrs
547 /// .into_iter()
548 /// .map(|attr| self.fold_attribute(attr))
549 /// .collect(),
550 /// left: Box::new(self.fold_expr(*node.left)),
551 /// op: self.fold_bin_op(node.op),
552 /// right: Box::new(self.fold_expr(*node.right)),
553 /// }
554 /// }
555 ///
556 /// /* ... */
557 /// # fn fold_attribute(&mut self, node: Attribute) -> Attribute;
558 /// # fn fold_expr(&mut self, node: Expr) -> Expr;
559 /// # fn fold_bin_op(&mut self, node: BinOp) -> BinOp;
560 /// }
561 /// ```
David Tolnay461d98e2018-01-07 11:07:19 -0800562 ///
563 /// *This module is available if Syn is built with the `"fold"` feature.*
Nika Layzella6f46c42017-10-26 15:26:16 -0400564 #[cfg(feature = "fold")]
565 pub mod fold;
David Tolnayf60f4262017-12-28 19:17:58 -0500566
David Tolnay0a0d78c2018-01-05 15:24:01 -0800567 #[cfg(any(feature = "full", feature = "derive"))]
David Tolnayf60f4262017-12-28 19:17:58 -0500568 #[path = "../gen_helper.rs"]
569 mod helper;
Nika Layzella6f46c42017-10-26 15:26:16 -0400570}
571pub use gen::*;
gnzlbg9ae88d82017-01-26 20:45:17 +0100572
David Tolnay456c9822018-08-25 08:09:46 -0400573// Not public API.
574#[doc(hidden)]
575pub mod export;
576
David Tolnay776f8e02018-08-24 22:32:10 -0400577#[cfg(feature = "parsing")]
David Tolnayb6254182018-08-25 08:44:54 -0400578mod lookahead;
579
580#[cfg(feature = "parsing")]
581pub mod parse;
582
David Tolnay776f8e02018-08-24 22:32:10 -0400583mod span;
584
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700585////////////////////////////////////////////////////////////////////////////////
586
David Tolnay55337722016-09-11 12:58:56 -0700587#[cfg(feature = "parsing")]
David Tolnayd641ee32018-08-30 17:11:09 -0700588use synom::Parser;
Ted Driggs054abbb2017-05-01 12:20:52 -0700589
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700590#[cfg(feature = "parsing")]
David Tolnayb6254182018-08-25 08:44:54 -0400591mod error;
David Tolnayc5ab8c62017-12-26 16:43:39 -0500592#[cfg(feature = "parsing")]
David Tolnayad4b2472018-08-25 08:25:24 -0400593use error::Error;
David Tolnayc5ab8c62017-12-26 16:43:39 -0500594
David Tolnayccab0be2018-01-06 22:24:47 -0800595/// Parse tokens of source code into the chosen syntax tree node.
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700596///
597/// This is preferred over parsing a string because tokens are able to preserve
598/// information about where in the user's code they were originally written (the
599/// "span" of the token), possibly allowing the compiler to produce better error
600/// messages.
601///
David Tolnayccab0be2018-01-06 22:24:47 -0800602/// This function parses a `proc_macro::TokenStream` which is the type used for
603/// interop with the compiler in a procedural macro. To parse a
604/// `proc_macro2::TokenStream`, use [`syn::parse2`] instead.
605///
606/// [`syn::parse2`]: fn.parse2.html
607///
hcpl4b72a382018-04-04 14:50:24 +0300608/// *This function is available if Syn is built with both the `"parsing"` and
609/// `"proc-macro"` features.*
David Tolnay461d98e2018-01-07 11:07:19 -0800610///
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700611/// # Examples
612///
David Tolnaybcf26022017-12-25 22:10:52 -0500613/// ```rust
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700614/// extern crate proc_macro;
615/// use proc_macro::TokenStream;
616///
617/// extern crate syn;
618///
619/// #[macro_use]
620/// extern crate quote;
621///
622/// use syn::DeriveInput;
623///
David Tolnaybcf26022017-12-25 22:10:52 -0500624/// # const IGNORE_TOKENS: &str = stringify! {
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700625/// #[proc_macro_derive(MyMacro)]
David Tolnaybcf26022017-12-25 22:10:52 -0500626/// # };
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700627/// pub fn my_macro(input: TokenStream) -> TokenStream {
628/// // Parse the tokens into a syntax tree
629/// let ast: DeriveInput = syn::parse(input).unwrap();
630///
631/// // Build the output, possibly using quasi-quotation
632/// let expanded = quote! {
633/// /* ... */
634/// };
635///
David Tolnaybcf26022017-12-25 22:10:52 -0500636/// // Convert into a token stream and return it
637/// expanded.into()
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700638/// }
David Tolnaybcf26022017-12-25 22:10:52 -0500639/// #
640/// # fn main() {}
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700641/// ```
David Tolnay278f9e32018-08-14 22:41:11 -0700642#[cfg(all(
643 not(all(target_arch = "wasm32", target_os = "unknown")),
644 feature = "parsing",
645 feature = "proc-macro"
646))]
David Tolnaye82a2b12018-08-30 16:31:10 -0700647pub fn parse<T: parse::Parse>(tokens: proc_macro::TokenStream) -> Result<T, Error> {
David Tolnayd641ee32018-08-30 17:11:09 -0700648 T::parse.parse(tokens)
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700649}
650
David Tolnayccab0be2018-01-06 22:24:47 -0800651/// Parse a proc-macro2 token stream into the chosen syntax tree node.
652///
653/// This function parses a `proc_macro2::TokenStream` which is commonly useful
654/// when the input comes from a node of the Syn syntax tree, for example the tts
655/// of a [`Macro`] node. When in a procedural macro parsing the
656/// `proc_macro::TokenStream` provided by the compiler, use [`syn::parse`]
657/// instead.
658///
659/// [`Macro`]: struct.Macro.html
660/// [`syn::parse`]: fn.parse.html
David Tolnay461d98e2018-01-07 11:07:19 -0800661///
662/// *This function is available if Syn is built with the `"parsing"` feature.*
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700663#[cfg(feature = "parsing")]
David Tolnaye82a2b12018-08-30 16:31:10 -0700664pub fn parse2<T: parse::Parse>(tokens: proc_macro2::TokenStream) -> Result<T, Error> {
David Tolnayd641ee32018-08-30 17:11:09 -0700665 T::parse.parse2(tokens)
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700666}
Alex Crichton954046c2017-05-30 21:49:42 -0700667
David Tolnayccab0be2018-01-06 22:24:47 -0800668/// Parse a string of Rust code into the chosen syntax tree node.
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700669///
David Tolnay461d98e2018-01-07 11:07:19 -0800670/// *This function is available if Syn is built with the `"parsing"` feature.*
671///
David Tolnay3f5b06f2018-01-11 21:02:00 -0800672/// # Hygiene
673///
674/// Every span in the resulting syntax tree will be set to resolve at the macro
675/// call site.
676///
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700677/// # Examples
678///
679/// ```rust
680/// extern crate syn;
681/// #
David Tolnay9174b972017-11-09 22:27:50 -0800682/// #
683/// # type Result<T> = std::result::Result<T, Box<std::error::Error>>;
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700684///
685/// use syn::Expr;
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700686///
687/// fn run() -> Result<()> {
688/// let code = "assert_eq!(u8::max_value(), 255)";
689/// let expr = syn::parse_str::<Expr>(code)?;
690/// println!("{:#?}", expr);
691/// Ok(())
692/// }
693/// #
694/// # fn main() { run().unwrap() }
695/// ```
696#[cfg(feature = "parsing")]
David Tolnaye82a2b12018-08-30 16:31:10 -0700697pub fn parse_str<T: parse::Parse>(s: &str) -> Result<T, Error> {
David Tolnayd641ee32018-08-30 17:11:09 -0700698 T::parse.parse_str(s)
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700699}
Alex Crichton954046c2017-05-30 21:49:42 -0700700
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700701// FIXME the name parse_file makes it sound like you might pass in a path to a
702// file, rather than the content.
703/// Parse the content of a file of Rust code.
704///
705/// This is different from `syn::parse_str::<File>(content)` in two ways:
706///
707/// - It discards a leading byte order mark `\u{FEFF}` if the file has one.
708/// - It preserves the shebang line of the file, such as `#!/usr/bin/env rustx`.
709///
710/// If present, either of these would be an error using `from_str`.
711///
Christopher Serr9727ef22018-02-03 16:42:12 +0100712/// *This function is available if Syn is built with the `"parsing"` and `"full"` features.*
David Tolnay461d98e2018-01-07 11:07:19 -0800713///
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700714/// # Examples
715///
716/// ```rust,no_run
717/// extern crate syn;
718/// #
David Tolnay9174b972017-11-09 22:27:50 -0800719/// #
720/// # type Result<T> = std::result::Result<T, Box<std::error::Error>>;
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700721///
722/// use std::fs::File;
723/// use std::io::Read;
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700724///
725/// fn run() -> Result<()> {
726/// let mut file = File::open("path/to/code.rs")?;
727/// let mut content = String::new();
728/// file.read_to_string(&mut content)?;
729///
730/// let ast = syn::parse_file(&content)?;
731/// if let Some(shebang) = ast.shebang {
732/// println!("{}", shebang);
733/// }
734/// println!("{} items", ast.items.len());
735///
736/// Ok(())
737/// }
738/// #
739/// # fn main() { run().unwrap() }
740/// ```
741#[cfg(all(feature = "parsing", feature = "full"))]
David Tolnayad4b2472018-08-25 08:25:24 -0400742pub fn parse_file(mut content: &str) -> Result<File, Error> {
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700743 // Strip the BOM if it is present
744 const BOM: &'static str = "\u{feff}";
745 if content.starts_with(BOM) {
746 content = &content[BOM.len()..];
David Tolnay35161ff2016-09-03 11:33:15 -0700747 }
Michael Layzell5e107ff2017-01-24 19:58:39 -0500748
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700749 let mut shebang = None;
750 if content.starts_with("#!") && !content.starts_with("#![") {
751 if let Some(idx) = content.find('\n') {
752 shebang = Some(content[..idx].to_string());
753 content = &content[idx..];
754 } else {
755 shebang = Some(content.to_string());
756 content = "";
757 }
Alex Crichton954046c2017-05-30 21:49:42 -0700758 }
David Tolnay0a8972b2017-02-27 02:10:01 -0800759
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700760 let mut file: File = parse_str(content)?;
761 file.shebang = shebang;
762 Ok(file)
Michael Layzell5e107ff2017-01-24 19:58:39 -0500763}
Alex Crichton259ee532017-07-14 06:51:02 -0700764
David Tolnayd641ee32018-08-30 17:11:09 -0700765/// Parse the input TokenStream of a macro, triggering a compile error if the
766/// tokens fail to parse.
767///
768/// # Intended usage
769///
770/// ```rust
771/// # extern crate proc_macro;
772/// # extern crate syn;
773/// #
774/// use proc_macro::TokenStream;
775/// use syn::parse_macro_input;
776/// use syn::parse::{Parse, ParseStream, Result};
777///
778/// struct MyMacroInput {
779/// /* ... */
780/// }
781///
782/// impl Parse for MyMacroInput {
783/// fn parse(input: ParseStream) -> Result<Self> {
784/// /* ... */
785/// # Ok(MyMacroInput {})
786/// }
787/// }
788///
789/// # const IGNORE: &str = stringify! {
790/// #[proc_macro]
791/// # };
792/// pub fn my_macro(tokens: TokenStream) -> TokenStream {
793/// let input = parse_macro_input!(tokens as MyMacroInput);
794///
795/// /* ... */
796/// # "".parse().unwrap()
797/// }
798/// #
799/// # fn main() {}
800/// ```
801#[cfg(feature = "proc-macro")]
802#[macro_export]
803macro_rules! parse_macro_input {
804 ($tokenstream:ident as $ty:ty) => {
805 match $crate::next::parse::<$ty>($tokenstream) {
806 $crate::export::Ok(data) => data,
807 $crate::export::Err(err) => {
808 return $crate::export::TokenStream::from(err.into_compile_error());
809 }
810 };
811 };
812}
813
David Tolnay278f9e32018-08-14 22:41:11 -0700814#[cfg(all(
815 any(feature = "full", feature = "derive"),
816 feature = "printing"
817))]
Alex Crichton259ee532017-07-14 06:51:02 -0700818struct TokensOrDefault<'a, T: 'a>(&'a Option<T>);
819
David Tolnay278f9e32018-08-14 22:41:11 -0700820#[cfg(all(
821 any(feature = "full", feature = "derive"),
822 feature = "printing"
823))]
Alex Crichton259ee532017-07-14 06:51:02 -0700824impl<'a, T> quote::ToTokens for TokensOrDefault<'a, T>
David Tolnay51382052017-12-27 13:46:21 -0500825where
826 T: quote::ToTokens + Default,
Alex Crichton259ee532017-07-14 06:51:02 -0700827{
Alex Crichtona74a1c82018-05-16 10:20:44 -0700828 fn to_tokens(&self, tokens: &mut proc_macro2::TokenStream) {
Alex Crichton259ee532017-07-14 06:51:02 -0700829 match *self.0 {
830 Some(ref t) => t.to_tokens(tokens),
831 None => T::default().to_tokens(tokens),
832 }
833 }
834}