blob: dc1ca341dc03824d016712c1a244814e467f2d4d [file] [log] [blame]
David Tolnay55535012018-01-05 16:39:23 -08001// Copyright 2018 Syn Developers
2//
3// Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
4// http://www.apache.org/licenses/LICENSE-2.0> or the MIT license
5// <LICENSE-MIT or http://opensource.org/licenses/MIT>, at your
6// option. This file may not be copied, modified, or distributed
7// except according to those terms.
8
David Tolnay5e84e972018-01-05 17:51:06 -08009//! Syn is a parsing library for parsing a stream of Rust tokens into a syntax
10//! tree of Rust source code.
11//!
David Tolnay6b889eb2018-09-01 18:12:17 -070012//! Currently this library is geared toward use in Rust procedural macros, but
13//! contains some APIs that may be useful more generally.
David Tolnay5e84e972018-01-05 17:51:06 -080014//!
15//! - **Data structures** — Syn provides a complete syntax tree that can
16//! represent any valid Rust source code. The syntax tree is rooted at
17//! [`syn::File`] which represents a full source file, but there are other
18//! entry points that may be useful to procedural macros including
19//! [`syn::Item`], [`syn::Expr`] and [`syn::Type`].
20//!
21//! - **Custom derives** — Of particular interest to custom derives is
22//! [`syn::DeriveInput`] which is any of the three legal input items to a
23//! derive macro. An example below shows using this type in a library that can
24//! derive implementations of a trait of your own.
25//!
David Tolnay6b889eb2018-09-01 18:12:17 -070026//! - **Parsing** — Parsing in Syn is built around [parser functions] with the
27//! signature `fn(ParseStream) -> Result<T>`. Every syntax tree node defined
28//! by Syn is individually parsable and may be used as a building block for
29//! custom syntaxes, or you may dream up your own brand new syntax without
30//! involving any of our syntax tree types.
David Tolnay5e84e972018-01-05 17:51:06 -080031//!
32//! - **Location information** — Every token parsed by Syn is associated with a
33//! `Span` that tracks line and column information back to the source of that
34//! token. These spans allow a procedural macro to display detailed error
35//! messages pointing to all the right places in the user's code. There is an
36//! example of this below.
37//!
38//! - **Feature flags** — Functionality is aggressively feature gated so your
39//! procedural macros enable only what they need, and do not pay in compile
40//! time for all the rest.
41//!
42//! [`syn::File`]: struct.File.html
43//! [`syn::Item`]: enum.Item.html
44//! [`syn::Expr`]: enum.Expr.html
45//! [`syn::Type`]: enum.Type.html
46//! [`syn::DeriveInput`]: struct.DeriveInput.html
David Tolnay6b889eb2018-09-01 18:12:17 -070047//! [parser functions]: parse/index.html
David Tolnay5e84e972018-01-05 17:51:06 -080048//!
49//! *Version requirement: Syn supports any compiler version back to Rust's very
50//! first support for procedural macros in Rust 1.15.0. Some features especially
51//! around error reporting are only available in newer compilers or on the
52//! nightly channel.*
53//!
54//! ## Example of a custom derive
55//!
56//! The canonical custom derive using Syn looks like this. We write an ordinary
57//! Rust function tagged with a `proc_macro_derive` attribute and the name of
58//! the trait we are deriving. Any time that derive appears in the user's code,
59//! the Rust compiler passes their data structure as tokens into our macro. We
60//! get to execute arbitrary Rust code to figure out what to do with those
61//! tokens, then hand some tokens back to the compiler to compile into the
62//! user's crate.
63//!
64//! [`TokenStream`]: https://doc.rust-lang.org/proc_macro/struct.TokenStream.html
65//!
66//! ```toml
67//! [dependencies]
David Tolnayb28acf32018-09-06 09:01:40 -070068//! syn = "0.15"
David Tolnay87003d02018-05-20 19:45:13 -070069//! quote = "0.6"
David Tolnay5e84e972018-01-05 17:51:06 -080070//!
71//! [lib]
72//! proc-macro = true
73//! ```
74//!
75//! ```rust
David Tolnaya1c98072018-09-06 08:58:10 -070076//! #[macro_use]
77//! extern crate quote;
78//! #[macro_use]
79//! extern crate syn;
80//!
81//! extern crate proc_macro;
82//!
David Tolnay5e84e972018-01-05 17:51:06 -080083//! use proc_macro::TokenStream;
David Tolnaya1c98072018-09-06 08:58:10 -070084//! use syn::DeriveInput;
David Tolnay5e84e972018-01-05 17:51:06 -080085//!
86//! # const IGNORE_TOKENS: &str = stringify! {
87//! #[proc_macro_derive(MyMacro)]
88//! # };
89//! pub fn my_macro(input: TokenStream) -> TokenStream {
90//! // Parse the input tokens into a syntax tree
David Tolnay6b889eb2018-09-01 18:12:17 -070091//! let input = parse_macro_input!(input as DeriveInput);
David Tolnay5e84e972018-01-05 17:51:06 -080092//!
93//! // Build the output, possibly using quasi-quotation
94//! let expanded = quote! {
95//! // ...
96//! };
97//!
98//! // Hand the output tokens back to the compiler
David Tolnay35b498e2018-09-01 20:10:40 -070099//! TokenStream::from(expanded)
David Tolnay5e84e972018-01-05 17:51:06 -0800100//! }
101//! #
102//! # fn main() {}
103//! ```
104//!
105//! The [`heapsize`] example directory shows a complete working Macros 1.1
106//! implementation of a custom derive. It works on any Rust compiler \>=1.15.0.
107//! The example derives a `HeapSize` trait which computes an estimate of the
108//! amount of heap memory owned by a value.
109//!
110//! [`heapsize`]: https://github.com/dtolnay/syn/tree/master/examples/heapsize
111//!
112//! ```rust
113//! pub trait HeapSize {
114//! /// Total number of bytes of heap memory owned by `self`.
115//! fn heap_size_of_children(&self) -> usize;
116//! }
117//! ```
118//!
119//! The custom derive allows users to write `#[derive(HeapSize)]` on data
120//! structures in their program.
121//!
122//! ```rust
123//! # const IGNORE_TOKENS: &str = stringify! {
124//! #[derive(HeapSize)]
125//! # };
126//! struct Demo<'a, T: ?Sized> {
127//! a: Box<T>,
128//! b: u8,
129//! c: &'a str,
130//! d: String,
131//! }
132//! ```
133//!
134//! ## Spans and error reporting
135//!
136//! The [`heapsize2`] example directory is an extension of the `heapsize`
137//! example that demonstrates some of the hygiene and error reporting properties
138//! of Macros 2.0. This example currently requires a nightly Rust compiler
139//! \>=1.24.0-nightly but we are working to stabilize all of the APIs involved.
140//!
141//! [`heapsize2`]: https://github.com/dtolnay/syn/tree/master/examples/heapsize2
142//!
143//! The token-based procedural macro API provides great control over where the
144//! compiler's error messages are displayed in user code. Consider the error the
145//! user sees if one of their field types does not implement `HeapSize`.
146//!
147//! ```rust
148//! # const IGNORE_TOKENS: &str = stringify! {
149//! #[derive(HeapSize)]
150//! # };
151//! struct Broken {
152//! ok: String,
153//! bad: std::thread::Thread,
154//! }
155//! ```
156//!
157//! In the Macros 1.1 string-based procedural macro world, the resulting error
158//! would point unhelpfully to the invocation of the derive macro and not to the
159//! actual problematic field.
160//!
161//! ```text
162//! error[E0599]: no method named `heap_size_of_children` found for type `std::thread::Thread` in the current scope
163//! --> src/main.rs:4:10
164//! |
165//! 4 | #[derive(HeapSize)]
166//! | ^^^^^^^^
167//! ```
168//!
169//! By tracking span information all the way through the expansion of a
170//! procedural macro as shown in the `heapsize2` example, token-based macros in
171//! Syn are able to trigger errors that directly pinpoint the source of the
172//! problem.
173//!
174//! ```text
175//! error[E0277]: the trait bound `std::thread::Thread: HeapSize` is not satisfied
176//! --> src/main.rs:7:5
177//! |
178//! 7 | bad: std::thread::Thread,
David Tolnayefff2ff2018-01-07 11:49:52 -0800179//! | ^^^^^^^^^^^^^^^^^^^^^^^^ the trait `HeapSize` is not implemented for `Thread`
David Tolnay5e84e972018-01-05 17:51:06 -0800180//! ```
181//!
David Tolnay6b889eb2018-09-01 18:12:17 -0700182//! ## Parsing a custom syntax
David Tolnay5e84e972018-01-05 17:51:06 -0800183//!
184//! The [`lazy-static`] example directory shows the implementation of a
185//! `functionlike!(...)` procedural macro in which the input tokens are parsed
David Tolnay6b889eb2018-09-01 18:12:17 -0700186//! using Syn's parsing API.
David Tolnay5e84e972018-01-05 17:51:06 -0800187//!
188//! [`lazy-static`]: https://github.com/dtolnay/syn/tree/master/examples/lazy-static
David Tolnay5e84e972018-01-05 17:51:06 -0800189//!
190//! The example reimplements the popular `lazy_static` crate from crates.io as a
191//! procedural macro.
192//!
193//! ```
194//! # macro_rules! lazy_static {
195//! # ($($tt:tt)*) => {}
196//! # }
197//! #
198//! lazy_static! {
199//! static ref USERNAME: Regex = Regex::new("^[a-z0-9_-]{3,16}$").unwrap();
200//! }
201//! ```
202//!
203//! The implementation shows how to trigger custom warnings and error messages
204//! on the macro input.
205//!
206//! ```text
207//! warning: come on, pick a more creative name
208//! --> src/main.rs:10:16
209//! |
210//! 10 | static ref FOO: String = "lazy_static".to_owned();
211//! | ^^^
212//! ```
213//!
214//! ## Debugging
215//!
216//! When developing a procedural macro it can be helpful to look at what the
217//! generated code looks like. Use `cargo rustc -- -Zunstable-options
218//! --pretty=expanded` or the [`cargo expand`] subcommand.
219//!
David Tolnay324db2d2018-01-07 11:51:09 -0800220//! [`cargo expand`]: https://github.com/dtolnay/cargo-expand
David Tolnay5e84e972018-01-05 17:51:06 -0800221//!
222//! To show the expanded code for some crate that uses your procedural macro,
223//! run `cargo expand` from that crate. To show the expanded code for one of
224//! your own test cases, run `cargo expand --test the_test_case` where the last
225//! argument is the name of the test file without the `.rs` extension.
226//!
227//! This write-up by Brandon W Maister discusses debugging in more detail:
228//! [Debugging Rust's new Custom Derive system][debugging].
229//!
230//! [debugging]: https://quodlibetor.github.io/posts/debugging-rusts-new-custom-derive-system/
231//!
232//! ## Optional features
233//!
234//! Syn puts a lot of functionality behind optional features in order to
235//! optimize compile time for the most common use cases. The following features
236//! are available.
237//!
238//! - **`derive`** *(enabled by default)* — Data structures for representing the
239//! possible input to a custom derive, including structs and enums and types.
240//! - **`full`** — Data structures for representing the syntax tree of all valid
241//! Rust source code, including items and expressions.
242//! - **`parsing`** *(enabled by default)* — Ability to parse input tokens into
243//! a syntax tree node of a chosen type.
244//! - **`printing`** *(enabled by default)* — Ability to print a syntax tree
245//! node as tokens of Rust source code.
246//! - **`visit`** — Trait for traversing a syntax tree.
David Tolnay34981cf2018-01-06 16:22:35 -0800247//! - **`visit-mut`** — Trait for traversing and mutating in place a syntax
David Tolnay5e84e972018-01-05 17:51:06 -0800248//! tree.
249//! - **`fold`** — Trait for transforming an owned syntax tree.
250//! - **`clone-impls`** *(enabled by default)* — Clone impls for all syntax tree
251//! types.
252//! - **`extra-traits`** — Debug, Eq, PartialEq, Hash impls for all syntax tree
253//! types.
hcpl4b72a382018-04-04 14:50:24 +0300254//! - **`proc-macro`** *(enabled by default)* — Runtime dependency on the
255//! dynamic library libproc_macro from rustc toolchain.
David Tolnay5e84e972018-01-05 17:51:06 -0800256
David Tolnay4cf7db82018-01-07 15:22:01 -0800257// Syn types in rustdoc of other crates get linked to here.
David Tolnayb28acf32018-09-06 09:01:40 -0700258#![doc(html_root_url = "https://docs.rs/syn/0.15.0")]
David Tolnay34071ba2018-05-20 20:00:41 -0700259#![cfg_attr(feature = "cargo-clippy", deny(clippy, clippy_pedantic))]
David Tolnay34071ba2018-05-20 20:00:41 -0700260// Ignored clippy lints.
David Tolnay94d2b792018-04-29 12:26:10 -0700261#![cfg_attr(
262 feature = "cargo-clippy",
263 allow(
David Tolnay4831ac62018-08-30 21:04:16 -0700264 block_in_if_condition_stmt,
David Tolnay0cec3e62018-07-21 09:08:30 -0700265 const_static_lifetime,
David Tolnay24b079d2018-08-27 08:28:10 -0700266 cyclomatic_complexity,
David Tolnay0cec3e62018-07-21 09:08:30 -0700267 doc_markdown,
David Tolnay24b079d2018-08-27 08:28:10 -0700268 eval_order_dependence,
David Tolnay0cec3e62018-07-21 09:08:30 -0700269 large_enum_variant,
270 match_bool,
David Tolnay6fb87462018-09-01 16:51:49 -0700271 never_loop,
David Tolnay0cec3e62018-07-21 09:08:30 -0700272 redundant_closure,
273 needless_pass_by_value,
274 redundant_field_names,
275 trivially_copy_pass_by_ref
David Tolnay94d2b792018-04-29 12:26:10 -0700276 )
277)]
David Tolnay34071ba2018-05-20 20:00:41 -0700278// Ignored clippy_pedantic lints.
279#![cfg_attr(
280 feature = "cargo-clippy",
281 allow(
David Tolnay0cec3e62018-07-21 09:08:30 -0700282 cast_possible_truncation,
283 cast_possible_wrap,
David Tolnayb1617752018-08-24 21:16:33 -0400284 empty_enum,
David Tolnay0cec3e62018-07-21 09:08:30 -0700285 if_not_else,
286 indexing_slicing,
287 items_after_statements,
David Tolnay151f92f2018-08-14 22:44:53 -0700288 shadow_unrelated,
David Tolnay0cec3e62018-07-21 09:08:30 -0700289 similar_names,
290 single_match_else,
291 stutter,
292 unseparated_literal_suffix,
293 use_self,
294 used_underscore_binding
David Tolnay34071ba2018-05-20 20:00:41 -0700295 )
296)]
David Tolnayad2836d2017-04-20 10:11:43 -0700297
David Tolnay278f9e32018-08-14 22:41:11 -0700298#[cfg(all(
299 not(all(target_arch = "wasm32", target_os = "unknown")),
300 feature = "proc-macro"
301))]
David Tolnay51382052017-12-27 13:46:21 -0500302extern crate proc_macro;
David Tolnay94d2b792018-04-29 12:26:10 -0700303extern crate proc_macro2;
David Tolnay570695e2017-06-03 16:15:13 -0700304extern crate unicode_xid;
Alex Crichtonccbb45d2017-05-23 10:58:24 -0700305
David Tolnay1cf80912017-12-31 18:35:12 -0500306#[cfg(feature = "printing")]
David Tolnay87d0b442016-09-04 11:52:12 -0700307extern crate quote;
308
Alex Crichton62a0a592017-05-22 13:58:53 -0700309#[macro_use]
310mod macros;
311
David Tolnay734079e2018-09-01 02:03:37 -0700312// Not public API.
David Tolnay852bff72018-08-27 08:24:02 -0700313#[cfg(feature = "parsing")]
David Tolnay734079e2018-09-01 02:03:37 -0700314#[doc(hidden)]
David Tolnay852bff72018-08-27 08:24:02 -0700315#[macro_use]
David Tolnay734079e2018-09-01 02:03:37 -0700316pub mod group;
David Tolnay852bff72018-08-27 08:24:02 -0700317
David Tolnayc5ab8c62017-12-26 16:43:39 -0500318#[macro_use]
David Tolnay32954ef2017-12-26 22:43:16 -0500319pub mod token;
David Tolnayc5ab8c62017-12-26 16:43:39 -0500320
David Tolnay4fb71232018-08-25 23:14:50 -0400321mod ident;
322pub use ident::Ident;
David Tolnaye303b7c2018-05-20 16:46:35 -0700323
David Tolnay3cfd1d32018-01-03 00:22:08 -0800324#[cfg(any(feature = "full", feature = "derive"))]
David Tolnayb79ee962016-09-04 09:39:20 -0700325mod attr;
David Tolnay3cfd1d32018-01-03 00:22:08 -0800326#[cfg(any(feature = "full", feature = "derive"))]
David Tolnayaaadd782018-01-06 22:58:13 -0800327pub use attr::{AttrStyle, Attribute, Meta, MetaList, MetaNameValue, NestedMeta};
David Tolnay35161ff2016-09-03 11:33:15 -0700328
David Tolnay3cfd1d32018-01-03 00:22:08 -0800329#[cfg(any(feature = "full", feature = "derive"))]
David Tolnayf38cdf62016-09-23 19:07:09 -0700330mod data;
David Tolnay3cfd1d32018-01-03 00:22:08 -0800331#[cfg(any(feature = "full", feature = "derive"))]
David Tolnayb57c8492018-05-05 00:32:04 -0700332pub use data::{
333 Field, Fields, FieldsNamed, FieldsUnnamed, Variant, VisCrate, VisPublic, VisRestricted,
334 Visibility,
335};
David Tolnayf38cdf62016-09-23 19:07:09 -0700336
David Tolnay3cfd1d32018-01-03 00:22:08 -0800337#[cfg(any(feature = "full", feature = "derive"))]
David Tolnayf4bbbd92016-09-23 14:41:55 -0700338mod expr;
David Tolnay3cfd1d32018-01-03 00:22:08 -0800339#[cfg(any(feature = "full", feature = "derive"))]
David Tolnayb57c8492018-05-05 00:32:04 -0700340pub use expr::{
David Tolnay02a9c6f2018-08-24 18:58:45 -0400341 Expr, ExprArray, ExprAssign, ExprAssignOp, ExprAsync, ExprBinary, ExprBlock, ExprBox,
342 ExprBreak, ExprCall, ExprCast, ExprClosure, ExprContinue, ExprField, ExprForLoop, ExprGroup,
David Tolnay9c119122018-09-01 18:47:02 -0700343 ExprIf, ExprInPlace, ExprIndex, ExprLet, ExprLit, ExprLoop, ExprMacro, ExprMatch,
David Tolnay02a9c6f2018-08-24 18:58:45 -0400344 ExprMethodCall, ExprParen, ExprPath, ExprRange, ExprReference, ExprRepeat, ExprReturn,
345 ExprStruct, ExprTry, ExprTryBlock, ExprTuple, ExprType, ExprUnary, ExprUnsafe, ExprVerbatim,
David Tolnay9c119122018-09-01 18:47:02 -0700346 ExprWhile, ExprYield, Index, Member,
David Tolnayb57c8492018-05-05 00:32:04 -0700347};
Michael Layzell734adb42017-06-07 16:58:31 -0400348
349#[cfg(feature = "full")]
David Tolnayb57c8492018-05-05 00:32:04 -0700350pub use expr::{
351 Arm, Block, FieldPat, FieldValue, GenericMethodArgument, Label, Local, MethodTurbofish, Pat,
352 PatBox, PatIdent, PatLit, PatMacro, PatPath, PatRange, PatRef, PatSlice, PatStruct, PatTuple,
353 PatTupleStruct, PatVerbatim, PatWild, RangeLimits, Stmt,
354};
David Tolnayf4bbbd92016-09-23 14:41:55 -0700355
David Tolnay3cfd1d32018-01-03 00:22:08 -0800356#[cfg(any(feature = "full", feature = "derive"))]
David Tolnayb79ee962016-09-04 09:39:20 -0700357mod generics;
David Tolnay3cfd1d32018-01-03 00:22:08 -0800358#[cfg(any(feature = "full", feature = "derive"))]
David Tolnayb57c8492018-05-05 00:32:04 -0700359pub use generics::{
360 BoundLifetimes, ConstParam, GenericParam, Generics, LifetimeDef, PredicateEq,
361 PredicateLifetime, PredicateType, TraitBound, TraitBoundModifier, TypeParam, TypeParamBound,
362 WhereClause, WherePredicate,
363};
David Tolnay278f9e32018-08-14 22:41:11 -0700364#[cfg(all(
365 any(feature = "full", feature = "derive"),
366 feature = "printing"
367))]
David Tolnayfd6bf5c2017-11-12 09:41:14 -0800368pub use generics::{ImplGenerics, Turbofish, TypeGenerics};
David Tolnay35161ff2016-09-03 11:33:15 -0700369
David Tolnayf38cdf62016-09-23 19:07:09 -0700370#[cfg(feature = "full")]
David Tolnayb79ee962016-09-04 09:39:20 -0700371mod item;
David Tolnayf38cdf62016-09-23 19:07:09 -0700372#[cfg(feature = "full")]
David Tolnayb57c8492018-05-05 00:32:04 -0700373pub use item::{
David Tolnay435c1782018-08-24 16:15:44 -0400374 ArgCaptured, ArgSelf, ArgSelfRef, FnArg, FnDecl, ForeignItem, ForeignItemFn, ForeignItemMacro,
375 ForeignItemStatic, ForeignItemType, ForeignItemVerbatim, ImplItem, ImplItemConst,
David Tolnaybb82ef02018-08-24 20:15:45 -0400376 ImplItemExistential, ImplItemMacro, ImplItemMethod, ImplItemType, ImplItemVerbatim, Item,
377 ItemConst, ItemEnum, ItemExistential, ItemExternCrate, ItemFn, ItemForeignMod, ItemImpl,
David Tolnayc6b04dd2018-08-30 23:22:51 -0700378 ItemMacro, ItemMacro2, ItemMod, ItemStatic, ItemStruct, ItemTrait, ItemTraitAlias, ItemType,
379 ItemUnion, ItemUse, ItemVerbatim, MethodSig, TraitItem, TraitItemConst, TraitItemMacro,
380 TraitItemMethod, TraitItemType, TraitItemVerbatim, UseGlob, UseGroup, UseName, UsePath,
381 UseRename, UseTree,
David Tolnayb57c8492018-05-05 00:32:04 -0700382};
David Tolnay35161ff2016-09-03 11:33:15 -0700383
David Tolnay631cb8c2016-11-10 17:16:41 -0800384#[cfg(feature = "full")]
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700385mod file;
David Tolnay631cb8c2016-11-10 17:16:41 -0800386#[cfg(feature = "full")]
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700387pub use file::File;
David Tolnay631cb8c2016-11-10 17:16:41 -0800388
David Tolnay63e3dee2017-06-03 20:13:17 -0700389mod lifetime;
390pub use lifetime::Lifetime;
391
David Tolnay3cfd1d32018-01-03 00:22:08 -0800392#[cfg(any(feature = "full", feature = "derive"))]
David Tolnayf4bbbd92016-09-23 14:41:55 -0700393mod lit;
David Tolnay3cfd1d32018-01-03 00:22:08 -0800394#[cfg(any(feature = "full", feature = "derive"))]
David Tolnayb57c8492018-05-05 00:32:04 -0700395pub use lit::{
396 FloatSuffix, IntSuffix, Lit, LitBool, LitByte, LitByteStr, LitChar, LitFloat, LitInt, LitStr,
397 LitVerbatim, StrStyle,
398};
David Tolnayf4bbbd92016-09-23 14:41:55 -0700399
David Tolnay3cfd1d32018-01-03 00:22:08 -0800400#[cfg(any(feature = "full", feature = "derive"))]
David Tolnayf4bbbd92016-09-23 14:41:55 -0700401mod mac;
David Tolnay3cfd1d32018-01-03 00:22:08 -0800402#[cfg(any(feature = "full", feature = "derive"))]
David Tolnayab919512017-12-30 23:31:51 -0500403pub use mac::{Macro, MacroDelimiter};
David Tolnayf4bbbd92016-09-23 14:41:55 -0700404
David Tolnay3cfd1d32018-01-03 00:22:08 -0800405#[cfg(any(feature = "full", feature = "derive"))]
David Tolnay0e837402016-12-22 17:25:55 -0500406mod derive;
David Tolnay3cfd1d32018-01-03 00:22:08 -0800407#[cfg(feature = "derive")]
David Tolnaye3d41b72017-12-31 15:24:00 -0500408pub use derive::{Data, DataEnum, DataStruct, DataUnion, DeriveInput};
David Tolnayf38cdf62016-09-23 19:07:09 -0700409
David Tolnay3cfd1d32018-01-03 00:22:08 -0800410#[cfg(any(feature = "full", feature = "derive"))]
David Tolnay3cb23a92016-10-07 23:02:21 -0700411mod op;
David Tolnay3cfd1d32018-01-03 00:22:08 -0800412#[cfg(any(feature = "full", feature = "derive"))]
David Tolnay3cb23a92016-10-07 23:02:21 -0700413pub use op::{BinOp, UnOp};
414
David Tolnay3cfd1d32018-01-03 00:22:08 -0800415#[cfg(any(feature = "full", feature = "derive"))]
David Tolnayb79ee962016-09-04 09:39:20 -0700416mod ty;
David Tolnay3cfd1d32018-01-03 00:22:08 -0800417#[cfg(any(feature = "full", feature = "derive"))]
David Tolnayb57c8492018-05-05 00:32:04 -0700418pub use ty::{
419 Abi, BareFnArg, BareFnArgName, ReturnType, Type, TypeArray, TypeBareFn, TypeGroup,
420 TypeImplTrait, TypeInfer, TypeMacro, TypeNever, TypeParen, TypePath, TypePtr, TypeReference,
421 TypeSlice, TypeTraitObject, TypeTuple, TypeVerbatim,
422};
David Tolnay056de302018-01-05 14:29:05 -0800423
424#[cfg(any(feature = "full", feature = "derive"))]
425mod path;
426#[cfg(any(feature = "full", feature = "derive"))]
David Tolnayb57c8492018-05-05 00:32:04 -0700427pub use path::{
David Tolnay9d0882a2018-09-01 19:49:14 -0700428 AngleBracketedGenericArguments, Binding, Constraint, GenericArgument,
429 ParenthesizedGenericArguments, Path, PathArguments, PathSegment, QSelf,
David Tolnayb57c8492018-05-05 00:32:04 -0700430};
Alex Crichtonccbb45d2017-05-23 10:58:24 -0700431
David Tolnay1b752fb2017-12-26 21:41:39 -0500432#[cfg(feature = "parsing")]
David Tolnaydfc886b2018-01-06 08:03:09 -0800433pub mod buffer;
David Tolnay94d304f2018-08-30 23:43:53 -0700434#[cfg(feature = "parsing")]
435pub mod ext;
David Tolnay94d2b792018-04-29 12:26:10 -0700436pub mod punctuated;
David Tolnay3779bb72018-08-26 18:46:07 -0700437#[cfg(all(
438 any(feature = "full", feature = "derive"),
439 feature = "extra-traits"
440))]
David Tolnaye0824032017-12-27 15:25:56 -0500441mod tt;
David Tolnayc5ab8c62017-12-26 16:43:39 -0500442
David Tolnaye83ef5a2018-01-11 15:18:36 -0800443// Not public API except the `parse_quote!` macro.
David Tolnay491680a2018-01-23 00:34:40 -0800444#[cfg(feature = "parsing")]
David Tolnaye83ef5a2018-01-11 15:18:36 -0800445#[doc(hidden)]
446pub mod parse_quote;
447
David Tolnay4d942b42018-01-02 22:14:04 -0800448#[cfg(all(feature = "parsing", feature = "printing"))]
David Tolnayf790b612017-12-31 18:46:57 -0500449pub mod spanned;
450
Nika Layzella6f46c42017-10-26 15:26:16 -0400451mod gen {
David Tolnayded2d682018-01-06 18:53:53 -0800452 /// Syntax tree traversal to walk a shared borrow of a syntax tree.
453 ///
454 /// Each method of the [`Visit`] trait is a hook that can be overridden to
455 /// customize the behavior when visiting the corresponding type of node. By
456 /// default, every method recursively visits the substructure of the input
457 /// by invoking the right visitor method of each of its fields.
458 ///
459 /// [`Visit`]: trait.Visit.html
460 ///
461 /// ```rust
462 /// # use syn::{Attribute, BinOp, Expr, ExprBinary};
463 /// #
464 /// pub trait Visit<'ast> {
465 /// /* ... */
466 ///
467 /// fn visit_expr_binary(&mut self, node: &'ast ExprBinary) {
468 /// for attr in &node.attrs {
469 /// self.visit_attribute(attr);
470 /// }
471 /// self.visit_expr(&*node.left);
472 /// self.visit_bin_op(&node.op);
473 /// self.visit_expr(&*node.right);
474 /// }
475 ///
476 /// /* ... */
477 /// # fn visit_attribute(&mut self, node: &'ast Attribute);
478 /// # fn visit_expr(&mut self, node: &'ast Expr);
479 /// # fn visit_bin_op(&mut self, node: &'ast BinOp);
480 /// }
481 /// ```
David Tolnay461d98e2018-01-07 11:07:19 -0800482 ///
483 /// *This module is available if Syn is built with the `"visit"` feature.*
Nika Layzella6f46c42017-10-26 15:26:16 -0400484 #[cfg(feature = "visit")]
485 pub mod visit;
David Tolnay55337722016-09-11 12:58:56 -0700486
David Tolnayded2d682018-01-06 18:53:53 -0800487 /// Syntax tree traversal to mutate an exclusive borrow of a syntax tree in
488 /// place.
489 ///
490 /// Each method of the [`VisitMut`] trait is a hook that can be overridden
491 /// to customize the behavior when mutating the corresponding type of node.
492 /// By default, every method recursively visits the substructure of the
493 /// input by invoking the right visitor method of each of its fields.
494 ///
495 /// [`VisitMut`]: trait.VisitMut.html
496 ///
497 /// ```rust
498 /// # use syn::{Attribute, BinOp, Expr, ExprBinary};
499 /// #
500 /// pub trait VisitMut {
501 /// /* ... */
502 ///
503 /// fn visit_expr_binary_mut(&mut self, node: &mut ExprBinary) {
504 /// for attr in &mut node.attrs {
505 /// self.visit_attribute_mut(attr);
506 /// }
507 /// self.visit_expr_mut(&mut *node.left);
508 /// self.visit_bin_op_mut(&mut node.op);
509 /// self.visit_expr_mut(&mut *node.right);
510 /// }
511 ///
512 /// /* ... */
513 /// # fn visit_attribute_mut(&mut self, node: &mut Attribute);
514 /// # fn visit_expr_mut(&mut self, node: &mut Expr);
515 /// # fn visit_bin_op_mut(&mut self, node: &mut BinOp);
516 /// }
517 /// ```
David Tolnay461d98e2018-01-07 11:07:19 -0800518 ///
519 /// *This module is available if Syn is built with the `"visit-mut"`
520 /// feature.*
David Tolnay9df02c42018-01-06 13:52:48 -0800521 #[cfg(feature = "visit-mut")]
Nika Layzella6f46c42017-10-26 15:26:16 -0400522 pub mod visit_mut;
Nika Layzell27726662017-10-24 23:16:35 -0400523
David Tolnayded2d682018-01-06 18:53:53 -0800524 /// Syntax tree traversal to transform the nodes of an owned syntax tree.
525 ///
526 /// Each method of the [`Fold`] trait is a hook that can be overridden to
527 /// customize the behavior when transforming the corresponding type of node.
528 /// By default, every method recursively visits the substructure of the
529 /// input by invoking the right visitor method of each of its fields.
530 ///
531 /// [`Fold`]: trait.Fold.html
532 ///
533 /// ```rust
534 /// # use syn::{Attribute, BinOp, Expr, ExprBinary};
535 /// #
536 /// pub trait Fold {
537 /// /* ... */
538 ///
539 /// fn fold_expr_binary(&mut self, node: ExprBinary) -> ExprBinary {
540 /// ExprBinary {
541 /// attrs: node.attrs
542 /// .into_iter()
543 /// .map(|attr| self.fold_attribute(attr))
544 /// .collect(),
545 /// left: Box::new(self.fold_expr(*node.left)),
546 /// op: self.fold_bin_op(node.op),
547 /// right: Box::new(self.fold_expr(*node.right)),
548 /// }
549 /// }
550 ///
551 /// /* ... */
552 /// # fn fold_attribute(&mut self, node: Attribute) -> Attribute;
553 /// # fn fold_expr(&mut self, node: Expr) -> Expr;
554 /// # fn fold_bin_op(&mut self, node: BinOp) -> BinOp;
555 /// }
556 /// ```
David Tolnay461d98e2018-01-07 11:07:19 -0800557 ///
558 /// *This module is available if Syn is built with the `"fold"` feature.*
Nika Layzella6f46c42017-10-26 15:26:16 -0400559 #[cfg(feature = "fold")]
560 pub mod fold;
David Tolnayf60f4262017-12-28 19:17:58 -0500561
David Tolnay0a0d78c2018-01-05 15:24:01 -0800562 #[cfg(any(feature = "full", feature = "derive"))]
David Tolnayf60f4262017-12-28 19:17:58 -0500563 #[path = "../gen_helper.rs"]
564 mod helper;
Nika Layzella6f46c42017-10-26 15:26:16 -0400565}
566pub use gen::*;
gnzlbg9ae88d82017-01-26 20:45:17 +0100567
David Tolnay456c9822018-08-25 08:09:46 -0400568// Not public API.
569#[doc(hidden)]
570pub mod export;
571
David Tolnay7fb11e72018-09-06 01:02:27 -0700572mod keyword;
David Tolnayb6254182018-08-25 08:44:54 -0400573
574#[cfg(feature = "parsing")]
David Tolnay7fb11e72018-09-06 01:02:27 -0700575mod lookahead;
Louis Kureuil Personc0beaf32018-09-05 00:12:43 +0200576
577#[cfg(feature = "parsing")]
David Tolnayb6254182018-08-25 08:44:54 -0400578pub mod parse;
579
David Tolnay776f8e02018-08-24 22:32:10 -0400580mod span;
581
David Tolnay64023912018-08-31 09:51:12 -0700582#[cfg(all(
583 any(feature = "full", feature = "derive"),
584 feature = "printing"
585))]
586mod print;
587
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700588////////////////////////////////////////////////////////////////////////////////
589
David Tolnayaa77a852018-08-31 11:15:10 -0700590#[cfg(any(feature = "parsing", feature = "full", feature = "derive"))]
David Tolnay94f06632018-08-31 10:17:17 -0700591#[allow(non_camel_case_types)]
David Tolnay10951d52018-08-31 10:27:39 -0700592struct private;
David Tolnay94f06632018-08-31 10:17:17 -0700593
594////////////////////////////////////////////////////////////////////////////////
595
David Tolnay55337722016-09-11 12:58:56 -0700596#[cfg(feature = "parsing")]
David Tolnayb6254182018-08-25 08:44:54 -0400597mod error;
David Tolnayc5ab8c62017-12-26 16:43:39 -0500598#[cfg(feature = "parsing")]
David Tolnayad4b2472018-08-25 08:25:24 -0400599use error::Error;
David Tolnayc5ab8c62017-12-26 16:43:39 -0500600
David Tolnayccab0be2018-01-06 22:24:47 -0800601/// Parse tokens of source code into the chosen syntax tree node.
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700602///
603/// This is preferred over parsing a string because tokens are able to preserve
604/// information about where in the user's code they were originally written (the
605/// "span" of the token), possibly allowing the compiler to produce better error
606/// messages.
607///
David Tolnayccab0be2018-01-06 22:24:47 -0800608/// This function parses a `proc_macro::TokenStream` which is the type used for
609/// interop with the compiler in a procedural macro. To parse a
610/// `proc_macro2::TokenStream`, use [`syn::parse2`] instead.
611///
612/// [`syn::parse2`]: fn.parse2.html
613///
hcpl4b72a382018-04-04 14:50:24 +0300614/// *This function is available if Syn is built with both the `"parsing"` and
615/// `"proc-macro"` features.*
David Tolnay461d98e2018-01-07 11:07:19 -0800616///
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700617/// # Examples
618///
David Tolnaybcf26022017-12-25 22:10:52 -0500619/// ```rust
David Tolnaya1c98072018-09-06 08:58:10 -0700620/// #[macro_use]
621/// extern crate quote;
622///
623/// extern crate proc_macro;
624/// extern crate syn;
625///
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700626/// use proc_macro::TokenStream;
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700627/// use syn::DeriveInput;
628///
David Tolnaybcf26022017-12-25 22:10:52 -0500629/// # const IGNORE_TOKENS: &str = stringify! {
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700630/// #[proc_macro_derive(MyMacro)]
David Tolnaybcf26022017-12-25 22:10:52 -0500631/// # };
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700632/// pub fn my_macro(input: TokenStream) -> TokenStream {
633/// // Parse the tokens into a syntax tree
634/// let ast: DeriveInput = syn::parse(input).unwrap();
635///
636/// // Build the output, possibly using quasi-quotation
637/// let expanded = quote! {
638/// /* ... */
639/// };
640///
David Tolnaybcf26022017-12-25 22:10:52 -0500641/// // Convert into a token stream and return it
642/// expanded.into()
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700643/// }
David Tolnaybcf26022017-12-25 22:10:52 -0500644/// #
645/// # fn main() {}
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700646/// ```
David Tolnay278f9e32018-08-14 22:41:11 -0700647#[cfg(all(
648 not(all(target_arch = "wasm32", target_os = "unknown")),
649 feature = "parsing",
650 feature = "proc-macro"
651))]
David Tolnaye82a2b12018-08-30 16:31:10 -0700652pub fn parse<T: parse::Parse>(tokens: proc_macro::TokenStream) -> Result<T, Error> {
David Tolnay80a914f2018-08-30 23:49:53 -0700653 parse::Parser::parse(T::parse, tokens)
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700654}
655
David Tolnayccab0be2018-01-06 22:24:47 -0800656/// Parse a proc-macro2 token stream into the chosen syntax tree node.
657///
658/// This function parses a `proc_macro2::TokenStream` which is commonly useful
659/// when the input comes from a node of the Syn syntax tree, for example the tts
660/// of a [`Macro`] node. When in a procedural macro parsing the
661/// `proc_macro::TokenStream` provided by the compiler, use [`syn::parse`]
662/// instead.
663///
664/// [`Macro`]: struct.Macro.html
665/// [`syn::parse`]: fn.parse.html
David Tolnay461d98e2018-01-07 11:07:19 -0800666///
667/// *This function is available if Syn is built with the `"parsing"` feature.*
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700668#[cfg(feature = "parsing")]
David Tolnaye82a2b12018-08-30 16:31:10 -0700669pub fn parse2<T: parse::Parse>(tokens: proc_macro2::TokenStream) -> Result<T, Error> {
David Tolnay80a914f2018-08-30 23:49:53 -0700670 parse::Parser::parse2(T::parse, tokens)
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700671}
Alex Crichton954046c2017-05-30 21:49:42 -0700672
David Tolnayccab0be2018-01-06 22:24:47 -0800673/// Parse a string of Rust code into the chosen syntax tree node.
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700674///
David Tolnay461d98e2018-01-07 11:07:19 -0800675/// *This function is available if Syn is built with the `"parsing"` feature.*
676///
David Tolnay3f5b06f2018-01-11 21:02:00 -0800677/// # Hygiene
678///
679/// Every span in the resulting syntax tree will be set to resolve at the macro
680/// call site.
681///
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700682/// # Examples
683///
684/// ```rust
David Tolnay9b00f652018-09-01 10:31:02 -0700685/// # extern crate syn;
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700686/// #
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700687/// use syn::Expr;
David Tolnay9b00f652018-09-01 10:31:02 -0700688/// use syn::parse::Result;
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700689///
690/// fn run() -> Result<()> {
691/// let code = "assert_eq!(u8::max_value(), 255)";
692/// let expr = syn::parse_str::<Expr>(code)?;
693/// println!("{:#?}", expr);
694/// Ok(())
695/// }
696/// #
697/// # fn main() { run().unwrap() }
698/// ```
699#[cfg(feature = "parsing")]
David Tolnaye82a2b12018-08-30 16:31:10 -0700700pub fn parse_str<T: parse::Parse>(s: &str) -> Result<T, Error> {
David Tolnay80a914f2018-08-30 23:49:53 -0700701 parse::Parser::parse_str(T::parse, s)
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700702}
Alex Crichton954046c2017-05-30 21:49:42 -0700703
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700704// FIXME the name parse_file makes it sound like you might pass in a path to a
705// file, rather than the content.
706/// Parse the content of a file of Rust code.
707///
708/// This is different from `syn::parse_str::<File>(content)` in two ways:
709///
710/// - It discards a leading byte order mark `\u{FEFF}` if the file has one.
711/// - It preserves the shebang line of the file, such as `#!/usr/bin/env rustx`.
712///
713/// If present, either of these would be an error using `from_str`.
714///
Christopher Serr9727ef22018-02-03 16:42:12 +0100715/// *This function is available if Syn is built with the `"parsing"` and `"full"` features.*
David Tolnay461d98e2018-01-07 11:07:19 -0800716///
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700717/// # Examples
718///
719/// ```rust,no_run
David Tolnay9b00f652018-09-01 10:31:02 -0700720/// # extern crate syn;
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700721/// #
David Tolnay9b00f652018-09-01 10:31:02 -0700722/// use std::error::Error;
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700723/// use std::fs::File;
724/// use std::io::Read;
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700725///
David Tolnay9b00f652018-09-01 10:31:02 -0700726/// fn run() -> Result<(), Box<Error>> {
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700727/// let mut file = File::open("path/to/code.rs")?;
728/// let mut content = String::new();
729/// file.read_to_string(&mut content)?;
730///
731/// let ast = syn::parse_file(&content)?;
732/// if let Some(shebang) = ast.shebang {
733/// println!("{}", shebang);
734/// }
735/// println!("{} items", ast.items.len());
736///
737/// Ok(())
738/// }
739/// #
740/// # fn main() { run().unwrap() }
741/// ```
742#[cfg(all(feature = "parsing", feature = "full"))]
David Tolnayad4b2472018-08-25 08:25:24 -0400743pub fn parse_file(mut content: &str) -> Result<File, Error> {
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700744 // Strip the BOM if it is present
745 const BOM: &'static str = "\u{feff}";
746 if content.starts_with(BOM) {
747 content = &content[BOM.len()..];
David Tolnay35161ff2016-09-03 11:33:15 -0700748 }
Michael Layzell5e107ff2017-01-24 19:58:39 -0500749
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700750 let mut shebang = None;
751 if content.starts_with("#!") && !content.starts_with("#![") {
752 if let Some(idx) = content.find('\n') {
753 shebang = Some(content[..idx].to_string());
754 content = &content[idx..];
755 } else {
756 shebang = Some(content.to_string());
757 content = "";
758 }
Alex Crichton954046c2017-05-30 21:49:42 -0700759 }
David Tolnay0a8972b2017-02-27 02:10:01 -0800760
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700761 let mut file: File = parse_str(content)?;
762 file.shebang = shebang;
763 Ok(file)
Michael Layzell5e107ff2017-01-24 19:58:39 -0500764}
Alex Crichton259ee532017-07-14 06:51:02 -0700765
David Tolnayd641ee32018-08-30 17:11:09 -0700766/// Parse the input TokenStream of a macro, triggering a compile error if the
767/// tokens fail to parse.
768///
David Tolnay8580f972018-09-01 17:53:03 -0700769/// Refer to the [`parse` module] documentation for more details about parsing
770/// in Syn.
771///
772/// [`parse` module]: parse/index.html
773///
David Tolnayd641ee32018-08-30 17:11:09 -0700774/// # Intended usage
775///
776/// ```rust
David Tolnaya1c98072018-09-06 08:58:10 -0700777/// #[macro_use]
778/// extern crate syn;
779///
780/// extern crate proc_macro;
781///
David Tolnayd641ee32018-08-30 17:11:09 -0700782/// use proc_macro::TokenStream;
David Tolnayd641ee32018-08-30 17:11:09 -0700783/// use syn::parse::{Parse, ParseStream, Result};
784///
785/// struct MyMacroInput {
786/// /* ... */
787/// }
788///
789/// impl Parse for MyMacroInput {
790/// fn parse(input: ParseStream) -> Result<Self> {
791/// /* ... */
792/// # Ok(MyMacroInput {})
793/// }
794/// }
795///
796/// # const IGNORE: &str = stringify! {
797/// #[proc_macro]
798/// # };
799/// pub fn my_macro(tokens: TokenStream) -> TokenStream {
800/// let input = parse_macro_input!(tokens as MyMacroInput);
801///
802/// /* ... */
803/// # "".parse().unwrap()
804/// }
805/// #
806/// # fn main() {}
807/// ```
808#[cfg(feature = "proc-macro")]
809#[macro_export]
810macro_rules! parse_macro_input {
811 ($tokenstream:ident as $ty:ty) => {
David Tolnay02465352018-08-30 23:38:00 -0700812 match $crate::parse::<$ty>($tokenstream) {
David Tolnayd641ee32018-08-30 17:11:09 -0700813 $crate::export::Ok(data) => data,
814 $crate::export::Err(err) => {
David Tolnay24e21f92018-09-06 02:08:07 -0700815 return $crate::export::TokenStream::from(err.to_compile_error());
David Tolnayd641ee32018-08-30 17:11:09 -0700816 }
817 };
818 };
819}