Package cssutils :: Module tokenize2 |
[hide private]
[frames] | no frames]

# Module tokenize2

source code

New CSS Tokenizer (a generator)

TODO: check selectors module tokenizer

test:

  - r'" ('

  - r' Û $4 m55 ¶666 ÿ777 7 ÿ7777'

  - r'#abc #123'

  - longer tokens before shorter

      1px -> 1

  - escapes

      color is one token? 1px = 1PX = 1px?

  - num: -0 are two tokens?

Version: $LastChangedRevision: 300 $

Date: $LastChangedDate: 2007-09-01 15:55:42 +0200 (Sa, 01 Sep 2007) $

Author: $LastChangedBy: cthedot $

Classes | [hide private]
| CSSProductions most attributes are set later
| Tokenizer (Tokenname, value, startline, startcolumn)

Imports: os, re, string, xml, cssutils, util, cssproductions.MACROS, cssproductions.PRODUCTIONS