How do lexical analyzers handle comment and escape sequences?

Comment and escape sequence (such as string literal) are very exceptional from regular symbolic representation.

It's hard to understand for me how does regular lexical analyzers tokenize them. How do lexical analyzers like lex, flex, or etc.. handle this kind of symbols? Is there a generic method? Or just case by case for each language?

5
задан Eonil 6 March 2011 в 11:51
поделиться