Crate cl_token

Token

Stores a component of a file as a TokenKind, some TokenData, and a line and column number

Re-exports

• pub use token::Token;
• pub use token_data::TokenData;
• pub use token_type::Punct;
• pub use token_type::TokenKind;

Modules

• token
  A Token contains a single unit of lexical information, and an optional bit of TokenData
• token_data
  Additional data stored within a Token, external to its TokenKind
• token_type
  Stores a Token’s lexical information