Source code documentation

pyexpander.lib module

The main pyexpander library.

class pyexpander.lib.BeginBlock(previous=None, filename=None)

Bases: Block

implements a $begin .. $end block.

This block is simply a variable scope, so it is derived from Block where the constructor is called with new_scope=True.

class pyexpander.lib.Block(previous=None, new_scope=False, filename=None, parse_list=None, external_definitions=None)

Bases: object

class that represents a block in the expander language.

Each block has a parent-pointer, which may be None. Blocks are used to represent syntactical blocks in the expander language. They manage the global variable dictionary and some more properties that are needed during execution of the program.

add_macro(name, macro_block)

add a new macro block.

eval_(st)

perform eval with the global variable dictionary of the block.

Here is an example:

>>> b= Block(parse_list=[])
>>> b.exec_("a=2")
>>> b.eval_("3*a")
6

exec_(st)

perform exec with the global variable dictionary of the block.

Here is an example:

>>> b= Block(parse_list=[])
>>> b.exec_("a=1")
>>> b["a"]
1
>>> b= Block(b,True)
>>> b["a"]
1
>>> b.exec_("a=2")
>>> b["a"]
2
>>> b.previous["a"]
1

export_symbols(lst)

appends items to the export_symbols list.

This list is used by the pop() method in order to copy values from the current global variable dictionary to the global variable dictionary of the previous block.

Here is an example:

>>> b= Block(parse_list=[])
>>> b.export_symbols(["a","b"])
>>> b.exported_syms
['a', 'b']
>>> b.export_symbols(["d","e"])
>>> b.exported_syms
['a', 'b', 'd', 'e']

extend(lst)

adds items to the list of expander functions or variables.

Here is an example:

>>> a=1
>>> b=2
>>> def t(x):
...   return x+1
...
>>> block= Block(parse_list=[],external_definitions=globals())
>>> block.extend(["a","b","t"])
>>> _pr_set(block.direct_vars)
set(['a', 'b'])
>>> _pr_set(block.direct_funcs)
set(['t'])

format_text(text, indent_start)

currently does indent one or more lines.

get_block_list()

returns all blocks of the list.

The list is returned with the oldest block first.

get_filename()

get the filename of the block, return ‘’ instead of None.

get_indent()

gets the indent.

parse_elm()

get the current parse element.

parse_loop()

loop across the items in the Block.

returns False if there is nothing more to parse in the current block.

pop()

removes the current block and returns the previous one.

Here is an example:

>>> b= Block(parse_list=[])
>>> b["a"]=1
>>> b["b"]=2
>>> b= Block(b,True)
>>> b["a"]=10
>>> b["b"]=20
>>> b.export_symbols(["a"])
>>> b= b.pop()
>>> b["a"]
10
>>> b["b"]
2

posmsg(msg=None, pos=None)

return a message together with a position.

This method is mainly used for error messages. We usually want to print filename and the line and column number together with a message. This method returns the given message string <msg> together with this additional information.

print_block_list()

print all blocks in the list.

The list is returned with the oldest block first.

Here is an example:

>>> b= Block(parse_list=[])
>>> print(b)
Block{
    has parent          : False
    filename            = None
    template            = None
    template_path       = None
    template_encoding   = None
    has template_parselist_cache: False
    exported_syms       = []
    direct_vars         = set([])
    direct_funcs        = set([])
    macros              : []
    new_scope           = False
    skip                = False
    safe_mode           = False
    indent              = 0
    new_parse_list      = True
    lst_pos             = -1
    start_pos           = 0
}
>>> b= Block(b,True)
>>> print(b)
Block{
    has parent          : True
    filename            = None
    template            = None
    template_path       = None
    template_encoding   = None
    has template_parselist_cache: False
    exported_syms       = []
    direct_vars         = set([])
    direct_funcs        = set([])
    macros              : []
    new_scope           = True
    skip                = False
    safe_mode           = False
    indent              = 0
    new_parse_list      = False
    lst_pos             = -1
    start_pos           = 0
}
>>> b.print_block_list()
Block{
    has parent          : False
    filename            = None
    template            = None
    template_path       = None
    template_encoding   = None
    has template_parselist_cache: False
    exported_syms       = []
    direct_vars         = set([])
    direct_funcs        = set([])
    macros              : []
    new_scope           = False
    skip                = False
    safe_mode           = False
    indent              = 0
    new_parse_list      = True
    lst_pos             = -1
    start_pos           = 0
}
Block{
    has parent          : True
    filename            = None
    template            = None
    template_path       = None
    template_encoding   = None
    has template_parselist_cache: False
    exported_syms       = []
    direct_vars         = set([])
    direct_funcs        = set([])
    macros              : []
    new_scope           = True
    skip                = False
    safe_mode           = False
    indent              = 0
    new_parse_list      = False
    lst_pos             = -1
    start_pos           = 0
}

set_indent(value)

sets the indent.

set_safemode(value)

sets the safe_mode.

set_substfile(filename, encoding, tp)

set substitution filename.

Used for the “$subst”, “$template” and “$pattern” commands.

setdefault(name, val)

return a value from globals, set to a default if it’s not defined.

Here are some examples:

>>> b= Block(parse_list=[])
>>> b["a"]
Traceback (most recent call last):
    ...
NameError: name 'a' is not defined at unknown position
>>> b.setdefault("a",10)
10
>>> b["a"]
10
>>> b["a"]=11
>>> b.setdefault("a",10)
11

str_block_list()

returns a string representation of all blocks in the list.

The list is returned with the oldest block first.

str_eval(st)

perform eval with the global variable dictionary of the block.

Here is an example:

>>> b= Block(parse_list=[])
>>> b.exec_("a=2")
>>> b.str_eval("3*a")
'6'

substfile_parselist(include_paths)

return the parselist for a given file.

This method manages a cache of that assigns filenames to parselists. This can speed things up if a file is included several times.

class pyexpander.lib.ForBlock(previous=None, new_scope=False, value_list=None, var_expr='')

Bases: Block

implements a $for .. $endfor block.

next_loop()

performs next loop.

returns:

True when the loop is not yet finished.

set_loop_var()

set the loop variable to a new value.

class pyexpander.lib.IfBlock(previous=None, condition=True)

Bases: Block

implements a $if .. $else .. $endif block.

An $if block never has a variable scope, so the base Block object is called with new_scope=False.

enter_elif(condition)

enter the “elif” part in the if..endif block.

enter_else()

this should be called when $else is encountered.

class pyexpander.lib.IncludeBlock(previous=None, new_scope=False, filename=None, encoding='UTF-8', include_paths=None)

Bases: Block

implements a $include(filename) block.

This block is simply a variable scope, so it is derived from Block where the constructor is called with new_scope=True.

class pyexpander.lib.MacBlock(previous=None, declaration_block=None, parameter_list=None)

Bases: Block

implements a $macro…$endmacro block.

pop()

override pop() from base class.

class pyexpander.lib.PatternBlock(previous=None, filename=None, include_paths=None, heading=None, lines=None)

Bases: Block

implements a $pattern(parameters) block.

This block is simply a variable scope, so it is derived from Block where the constructor is called with new_scope=True.

def_vars()

define all the given variables in the PatternBlock.

pop()

removes the current block and returns the previous one.

Here is an example:

>>> b= Block(parse_list=[])
>>> b["a"]=1
>>> b["b"]=2
>>> b= Block(b,True)
>>> b["a"]=10
>>> b["b"]=20
>>> b.export_symbols(["a"])
>>> b= b.pop()
>>> b["a"]
10
>>> b["b"]
2

class pyexpander.lib.ResultText

Bases: object

basically a list of strings with a current column property.

append(text)

append some text.

column()

return current column.

static current_column(st)

find current column if the string is printed.

Note: With a string ending with ‘n’ this returns 1.

Here are some examples: >>> ResultText.current_column(“”) -1 >>> ResultText.current_column(“ab”) -1 >>> ResultText.current_column(”nab”) 3 >>> ResultText.current_column(”nabn”) 1 >>> ResultText.current_column(”nabna”) 2

list_()

return internal list.

class pyexpander.lib.SubstBlock(previous=None, filename=None, include_paths=None, external_definitions=None)

Bases: Block

implements a $subst(parameters) block.

This block is simply a variable scope, so it is derived from Block where the constructor is called with new_scope=True.

class pyexpander.lib.WhileBlock(previous=None, new_scope=False, while_expr='')

Bases: Block

implements a $while .. $endwhile block.

next_loop()

performs next loop.

returns:

True when the loop is not yet finished.

pyexpander.lib.expand(st, filename=None, external_definitions=None, allow_nobracket_vars=False, auto_continuation=False, auto_indent=False, include_paths=None, output_encoding='UTF-8', print_mode='full')

Get a string, expand the text in it and print it.

args:

• st (str): The string that is to be expaned.

• filename (str): The filename, if given, is included in possible error messages.

• external_definitions (dict): A dict with items to import to the globals() dictionary.

• allow_nobracket_vars (bool): If True, allow variables in the form $VAR instead of $(VAR).

• auto_continuation (bool): If True, remove newline at the end of lines with a command. This works like having an ‘' at the end of each line with a command.

• auto_indent (bool): If True, indent the contents of macros to the same level as the macro invocation.

• include_paths (list): A list of paths that are searched for the $include command.

• output_encoding: encoding used to create output data

• print_mode: one of 3 possible strings,

  • full: normal printing

  • repr: print the list of strings in python “repr” format, this is for debugging only.

  • none: do not print anything

returns:

• The internal globals() dictionary.

• The file_deps dict with file dependencies

pyexpander.lib.expandFile(filename, encoding='UTF-8', external_definitions=None, allow_nobracket_vars=False, auto_continuation=False, auto_indent=False, include_paths=None, no_stdin_warning=False, output_encoding='UTF-8', print_mode='full')

Get a filename, expand the text in it and print it.

args:

• filename (str): The name of the file

• encoding: encoding of the file

• external_definitions (dict): A dict with items to import to the globals() dictionary.

• allow_nobracket_vars (bool): If True, allow variables in the form $VAR instead of $(VAR).

• auto_continuation (bool): If True, remove newline at the end of lines with a command. This works like having an ‘’ at the end of each line with a command.

• auto_indent (bool): If True, indent the contents of macros to the same level as the macro invocation.

• include_paths (list): A list of paths that are searched for the $include command.

• no_stdin_warning (bool): If True, print short message on stderr when the program is waiting on input from stdin.

• output_encoding: encoding used to create output data

• print_mode: one of 3 possible strings,

  • full: normal printing

  • repr: print the list of strings in python “repr” format, this is for debugging only.

  • none: do not print anything

returns:

• The internal globals() dictionary.

• The file_deps dict with file dependencies

pyexpander.lib.expandToStr(st, filename=None, external_definitions=None, allow_nobracket_vars=False, auto_continuation=False, auto_indent=False, include_paths=None)

Get a string, expand the text in it and return it as a string.

args:

• st (str): The string that is to be expaned.

• filename (str): The filename, if given, is included in possible error messages.

• external_definitions (dict): A dict with items to import to the globals() dictionary.

• allow_nobracket_vars (bool): If True, allow variables in the form $VAR instead of $(VAR).

• auto_continuation (bool): If True, remove newline at the end of lines with a command. This works like having an ‘’ at the end of each line with a command.

• auto_indent (bool): If True, indent the contents of macros to the same level as the macro invocation.

• include_paths (list): A list of paths that are searched for the $include command.

returns a tuple containing:

• The expanded text as a single string.

• The internal globals() dictionary.

• The file_deps dict with file dependencies

pyexpander.lib.find_file(filename, include_paths)

find a file in a list of include paths.

include_paths MUST CONTAIN “” in order to search the local directory.

pyexpander.lib.keyword_check(identifiers)

indentifiers must not be identical to keywords.

This function may raise an exception.

pyexpander.lib.merge_dependecies(deps, new_deps)

merge contents of two dependency dicts.

These are the “file_deps” as they are returned by functions like processToList.

pyexpander.lib.one_or_two_strings(arg)

arg must be a single string or a tuple of two strings.

pyexpander.lib.parseFile(filename, encoding, no_stdin_warning)

parse a file.

pyexpander.lib.parseString(st)

parse a string.

pyexpander.lib.print_dependencies(deps)

print dependencies in make compatible format.

These are the “file_deps” as they are returned by functions like processToList.

pyexpander.lib.processToList(parse_list, filename=None, external_definitions=None, allow_nobracket_vars=False, auto_continuation=False, auto_indent=False, include_paths=None)

Expand a parse list to a list of strings.

args:

parse_list: A parse list created by parseString().

filename (str): The filename, if given, is included in possible error

messages.

external_definitions (dict): A dict with items to import to the

globals() dictionary.

allow_nobracket_vars (bool): If True, allow variables in the form $VAR

instead of $(VAR).

auto_continuation (bool): If True, remove newline at the end of lines

with a command. This works like having an ‘’ at the end of each line with a command.

auto_indent (bool): If True, indent the contents of macros to the same

level as the macro invocation.

include_paths (list): A list of paths that are searched for the

$include command.

returns a tuple containing:

• The expanded text as a list of strings

• The internal globals() dictionary.

• The file_deps dict with file dependencies

pyexpander.lib.processToPrint(parse_list, filename=None, external_definitions=None, allow_nobracket_vars=False, auto_continuation=False, auto_indent=False, include_paths=None, output_encoding='UTF-8', print_mode='full')

Gets a parse list, expand the text in it and print it.

args:

• parse_list: A parse list created by parseString().

• filename (str): The filename, if given, is included in possible error messages.

• external_definitions (dict): A dict with items to import to the globals() dictionary.

• allow_nobracket_vars (bool): If True, allow variables in the form $VAR instead of $(VAR).

• auto_continuation (bool): If True, remove newline at the end of lines with a command. This works like having an ‘’ at the end of each line with a command.

• auto_indent (bool): If True, indent the contents of macros to the same level as the macro invocation.

• include_paths (list): A list of paths that are searched for the $include command.

• output_encoding: encoding used to create output data

• print_mode: one of 3 possible strings,

  • full: normal printing

  • repr: print the list of strings in python “repr” format, this is for debugging only.

  • none: do not print anything

returns:

• The internal globals() dictionary.

• The file_deps dict with file dependencies

pyexpander.lib.test_encoding(encoding)

test if an encoding is known.

raises (by encode() method) a LookupError exception in case of an error.

pyexpander.parser module

implement the parser for pyexpander.

class pyexpander.parser.IndexedString(st)

Bases: object

a string together with row column information.

Here is an example:

>>> txt='''01234
... 67
... 9abcd'''
>>> l=IndexedString(txt)
>>> l.rowcol(0)
(1, 1)
>>> l.rowcol(1)
(1, 2)
>>> l.rowcol(4)
(1, 5)
>>> l.rowcol(5)
(1, 6)
>>> l.rowcol(6)
(2, 1)
>>> l.rowcol(7)
(2, 2)
>>> l.rowcol(8)
(2, 3)
>>> l.rowcol(9)
(3, 1)
>>> l.rowcol(13)
(3, 5)
>>> l.rowcol(14)
(3, 6)
>>> l.rowcol(16)
(3, 8)

rowcol(pos)

calculate (row,column) from a string position.

st()

return the raw string.

exception pyexpander.parser.ParseException(value, pos=None, rowcol=None)

Bases: Exception

used for Exceptions in this module.

class pyexpander.parser.ParsedCommand(idxst, start, end, ident)

Bases: ParsedItem

class of a pyexpander command with arguments.

A command has the form “$name(argument1, argument2, …)”.

args()

return the arguments of the command.

class pyexpander.parser.ParsedComment(idxst, start, end)

Bases: ParsedItem

class of a parsed comment.

A comment in pyexpander starts with ‘$#’.

class pyexpander.parser.ParsedEval(idxst, start, end)

Bases: ParsedItem

class of an pyexpander expression.

A pyexpander expression has the form “$(expression)” e.g. “$(a+1)”. This is different from ParsedVar where the string within the brackets is a simple identifier.

class pyexpander.parser.ParsedItem(idxst, start, end)

Bases: object

base class of parsed items.

end()

return the end of the ParsedItem in the source string.

positions()

return start and end of ParsedItem in the source string.

rowcol(pos=None)

calculate (row,column) from a string position.

start()

return the start of the ParsedItem in the source string.

string()

return the string that represents the ParsedItem.

class pyexpander.parser.ParsedLiteral(idxst, start, end)

Bases: ParsedItem

class of a parsed literal.

A literal is a substring in the input that shouldn’t be modified by pyexpander.

class pyexpander.parser.ParsedPureCommand(idxst, start, end)

Bases: ParsedItem

class of a pyexpander command without arguments.

A pure command has the form “$name”. Such a command has no arguments which would be enclosed in round brackets immediately following the name.

class pyexpander.parser.ParsedVar(idxst, start, end)

Bases: ParsedItem

class of a parsed variable.

A variable in pyexpander has the form “$(identifier)”.

pyexpander.parser.change_linesep(sep)

change line separator, this is here just for tests.

pyexpander.parser.parseAll(idxst, pos)

parse everything.

>>> def test(st,pos):
...     idxst= IndexedString(st)
...     pprint(parseAll(idxst,pos))
...

>>> test("abc",0)
('ParsedLiteral', (0, 2), 'abc')
>>> test("abc$xyz",0)
('ParsedLiteral', (0, 2), 'abc')
('ParsedPureCommand', (4, 6), 'xyz')
>>> test("abc${xyz}efg",0)
('ParsedLiteral', (0, 2), 'abc')
('ParsedPureCommand', (5, 7), 'xyz')
('ParsedLiteral', (9, 11), 'efg')
>>> test("abc$xyz(2*4)",0)
('ParsedLiteral', (0, 2), 'abc')
('ParsedCommand', (8, 10), '2*4', 'xyz')
>>> test("abc$(2*4)ab",0)
('ParsedLiteral', (0, 2), 'abc')
('ParsedEval', (5, 7), '2*4')
('ParsedLiteral', (9, 10), 'ab')
>>> test("abc\\$(2*4)ab",0)
('ParsedLiteral', (0, 2), 'abc')
('ParsedLiteral', (4, 4), '$')
('ParsedLiteral', (5, 11), '(2*4)ab')
>>> test("ab$func(1+2)\\\nnew line",0)
('ParsedLiteral', (0, 1), 'ab')
('ParsedCommand', (8, 10), '1+2', 'func')
('ParsedLiteral', (14, 21), 'new line')
>>> test("ab$func(1+2)\nnew line",0)
('ParsedLiteral', (0, 1), 'ab')
('ParsedCommand', (8, 10), '1+2', 'func')
('ParsedLiteral', (12, 20), '\nnew line')
>>> test("ab$(xyz)(56)",0)
('ParsedLiteral', (0, 1), 'ab')
('ParsedVar', (4, 6), 'xyz')
('ParsedLiteral', (8, 11), '(56)')

>>> test(r'''
... Some text with a macro: $(xy)
... an escaped dollar: \$(xy)
... a macro within letters: abc${xy}def
... a pyexpander command structure:
... $if(a=1)
... here
... $else
... there
... $endif
... now a continued\
... line
... from here:$# the rest is a comment
... now an escaped continued\\
... line
... ''',0)
('ParsedLiteral', (0, 24), '\nSome text with a macro: ')
('ParsedVar', (27, 28), 'xy')
('ParsedLiteral', (30, 49), '\nan escaped dollar: ')
('ParsedLiteral', (51, 51), '$')
('ParsedLiteral', (52, 83), '(xy)\na macro within letters: abc')
('ParsedPureCommand', (86, 87), 'xy')
('ParsedLiteral', (89, 124), 'def\na pyexpander command structure:\n')
('ParsedCommand', (129, 131), 'a=1', 'if')
('ParsedLiteral', (133, 138), '\nhere\n')
('ParsedPureCommand', (140, 143), 'else')
('ParsedLiteral', (144, 150), '\nthere\n')
('ParsedPureCommand', (152, 156), 'endif')
('ParsedLiteral', (157, 172), '\nnow a continued')
('ParsedLiteral', (175, 189), 'line\nfrom here:')
('ParsedComment', (192, 214), ' the rest is a comment\n')
('ParsedLiteral', (215, 238), 'now an escaped continued')
('ParsedLiteral', (239, 239), '\\')
('ParsedLiteral', (241, 246), '\nline\n')

pyexpander.parser.parseBackslash(idxst, pos)

parses a backslash.

>>> import os
>>> def test(st,pos,sep=None):
...     if sep:
...         change_linesep(sep)
...     idxst= IndexedString(st)
...     (p,elm)= parseBackslash(idxst,pos)
...     print("Parsed: %s" % elm)
...     print("rest of string:%s" % repr(st[p:]))
...     change_linesep(os.linesep)
...
>>> test(r"\abc",0)
Parsed: ('ParsedLiteral', (0, 0), '\\')
rest of string:'abc'
>>> test("\\",0)
Parsed: ('ParsedLiteral', (0, 0), '\\')
rest of string:''
>>> test("\\\rab",0,"\r")
Parsed: None
rest of string:'ab'
>>> test("\\\rab",0,"\n")
Parsed: ('ParsedLiteral', (0, 0), '\\')
rest of string:'\rab'
>>> test("\\\rab",0,"\r\n")
Parsed: ('ParsedLiteral', (0, 0), '\\')
rest of string:'\rab'
>>> test("\\\nab",0,"\r")
Parsed: ('ParsedLiteral', (0, 0), '\\')
rest of string:'\nab'
>>> test("\\\nab",0,"\n")
Parsed: None
rest of string:'ab'
>>> test("\\\nab",0,"\r\n")
Parsed: ('ParsedLiteral', (0, 0), '\\')
rest of string:'\nab'
>>> test("\\\r\nab",0,"\r")
Parsed: None
rest of string:'\nab'
>>> test("\\\r\nab",0,"\n")
Parsed: ('ParsedLiteral', (0, 0), '\\')
rest of string:'\r\nab'
>>> test("\\\r\nab",0,"\r\n")
Parsed: None
rest of string:'ab'

pyexpander.parser.parseBracketed(idxst, pos)

parse an identifier in curly brackets.

Here are some examples:

>>> def test(st,pos):
...     idxst= IndexedString(st)
...     (a,b)= parseBracketed(idxst,pos)
...     print(st[a:b])
...
>>> test(r'{abc}',0)
{abc}
>>> test(r'{ab8c}',0)
{ab8c}
>>> test(r'{c}',0)
{c}
>>> test(r'{}',0)
Traceback (most recent call last):
    ...
ParseException: command enclosed in curly brackets at line 1, col 1
>>> test(r'{abc',0)
Traceback (most recent call last):
    ...
ParseException: command enclosed in curly brackets at line 1, col 1
>>> test(r'x{ab8c}',1)
{ab8c}

pyexpander.parser.parseCode(idxst, pos)

parse python code, it MUST start with a ‘(‘.

Here are some examples:

>>> def test(st,pos):
...     idxst= IndexedString(st)
...     (a,b)= parseCode(idxst,pos)
...     print(st[a:b])
...
>>> test(r'(a+b)',0)
(a+b)
>>> test(r'(a+(b*c))',0)
(a+(b*c))
>>> test(r'(a+(b*c)+")")',0)
(a+(b*c)+")")
>>> test(r"(a+(b*c)+''')''')",0)
(a+(b*c)+''')''')
>>> test(r"(a+(b*c)+''')'''+# comment )\n)",0)
Traceback (most recent call last):
    ...
ParseException: end of bracket expression not found at line 1, col 1
>>>
>>> test("(a+(b*c)+''')'''+# comment )\n)",0)
(a+(b*c)+''')'''+# comment )
)

pyexpander.parser.parseComment(idxst, pos)

parse a python comment.

Here are some examples:

>>> import os
>>> def test(st,pos,sep=None):
...     if sep:
...         change_linesep(sep)
...     idxst= IndexedString(st)
...     (a,b)= parseComment(idxst,pos)
...     print(repr(st[a:b]))
...     change_linesep(os.linesep)
...
>>> test("#abc",0)
'#abc'
>>> test("#abc\nef",0,"\n")
'#abc\n'
>>> test("#abc\r\nef",0,"\r\n")
'#abc\r\n'
>>> test("xy#abc",2)
'#abc'
>>> test("xy#abc\nef",2,"\n")
'#abc\n'
>>> test("xy#abc\nef",3)
Traceback (most recent call last):
    ...
ParseException: start of comment not found at line 1, col 4

pyexpander.parser.parseDollar(idxst, pos)

parse things that follow a dollar.

Here are some examples:

>>> def test(st,pos):
...   idxst= IndexedString(st)
...   (p,elm)= parseDollar(idxst,pos)
...   print("Parsed: %s" % elm)
...   print("rest of string:%s" % st[p:])
...
>>> test("$abc",0)
Parsed: ('ParsedPureCommand', (1, 3), 'abc')
rest of string:
>>> test("$abc%&/",0)
Parsed: ('ParsedPureCommand', (1, 3), 'abc')
rest of string:%&/
>>> test("$abc(2*3)",0)
Parsed: ('ParsedCommand', (5, 7), '2*3', 'abc')
rest of string:
>>> test(" $abc(2*sin(x))",1)
Parsed: ('ParsedCommand', (6, 13), '2*sin(x)', 'abc')
rest of string:
>>> test(" $abc(2*sin(x))bn",1)
Parsed: ('ParsedCommand', (6, 13), '2*sin(x)', 'abc')
rest of string:bn
>>> test(" $# a comment\nnew line",1)
Parsed: ('ParsedComment', (3, 13), ' a comment\n')
rest of string:new line
>>> test("$(abc)",0)
Parsed: ('ParsedVar', (2, 4), 'abc')
rest of string:
>>> test("$(abc*2)",0)
Parsed: ('ParsedEval', (2, 6), 'abc*2')
rest of string:
>>> test(" $(2*x(y))abc",1)
Parsed: ('ParsedEval', (3, 8), '2*x(y)')
rest of string:abc
>>> test(" $   (2*x(y))abc",1)
Parsed: ('ParsedEval', (6, 11), '2*x(y)')
rest of string:abc

pyexpander.parser.parseStringLiteral(idxst, pos)

parse a python string literal.

returns 2 numbers, the index where the string starts and the index of the first character after the string

Here are some examples:

>>> def test(st,pos):
...     idxst= IndexedString(st)
...     (a,b)= parseStringLiteral(idxst,pos)
...     print(st[a:b])
...

>>> test(r'''"abc"''',0)
"abc"

>>> test("'''ab'c'd'''",0)
'''ab'c'd'''
>>> test("'''ab'cd''''",0)
'''ab'cd'''

>>> test(r'''F"abc"''',0)
F"abc"
>>> test(r'''xF"abc"''',1)
F"abc"
>>> test(r'''xFr"abc"''',1)
Fr"abc"

>>> test(r'''xFr"ab\\"c"''',1)
Fr"ab\\"

>>> test(r'''xFr"ab\"c"''',1)
Fr"ab\"c"
>>> test(r'''xFr"ab\"c"''',0)
Traceback (most recent call last):
    ...
ParseException: start of string expected at line 1, col 1
>>> test(r'''"ab''',0)
Traceback (most recent call last):
    ...
ParseException: end of string not found at line 1, col 1
>>> test(r"'''ab'",0)
Traceback (most recent call last):
    ...
ParseException: end of string not found at line 1, col 1
>>> test(r'''"ab\"''',0)
Traceback (most recent call last):
    ...
ParseException: end of string not found at line 1, col 1

pyexpander.parser.pprint(parselist)

pretty print a parselist.

pyexpander.parser.scanPyIdentList(st)

scan a list of python identifiers.

Here are some examples:

>>> scanPyIdentList("a,b")
['a', 'b']
>>> scanPyIdentList("a,b.d, c")
['a', 'b.d', 'c']
>>> scanPyIdentList("a,b.d, c&")
Traceback (most recent call last):
    ...
ParseException: list of python identifiers expected

pyexpander.parser.scanPyIn(st)

scan a python “in” statement.

Here are some examples:

>>> scanPyIn(" (a,b) in k.items() ")
('(a,b)', 'in', 'k.items()')

expander script

expander.py: this is the pyexpander application.

expander.check_encoding(encoding)

check if the encoding name is valid.

expander.main()

The main function.

parse the command-line options and perform the command

expander.print_summary()

print a short summary of the scripts function.

expander.process_files(options, args)

process all the command line options.

expander.script_shortname()

return the name of this script without a path component.