API reference¶

Module contents¶

A speedy LookML parser and serializer implemented in pure Python.

lkml.cli()¶: Command-line entry point for lkml.

lkml.dump(obj: dict, file_object: IO | None = None) → str | None¶

Serialize a Python dictionary into LookML.

Parameters:

obj – The Python dictionary to be serialized to LookML
file_object – An optional file object to save the LookML string to

Returns:

A LookML string if no file_object is passed

lkml.load(stream: str | IO) → dict¶

Parse LookML into a Python dictionary.

Parameters:: stream – File object or string containing LookML to be parsed
Raises:: TypeError – If stream is neither a string or a file object

lkml.parse(text: str) → DocumentNode¶

Parse LookML into a parse tree.

Parameters:: text – The LookML string to be parsed.
Returns:: A document node, the root of the parse tree.

lkml.parse_args(args: Sequence) → Namespace¶: Parse command-line arguments.

lkml.keys module¶

Defines constant sequences of LookML keys and helper methods.

lkml.keys.pluralize(key: str) → str¶: Converts a singular key like “explore” to a plural key, e.g. ‘explores’.

lkml.keys.singularize(key: str) → str¶: Converts a plural key like “explores” to a singular key, e.g. ‘explore’.

lkml.lexer module¶

Splits a LookML string into a sequence of tokens.

class lkml.lexer.Lexer(text: str)¶

Splits a LookML string into a sequence of tokens.

text¶: Raw LookML to parse, padded with null character to denote end of stream

index¶: Position of lexer in characters as it traverses the text

tokens¶: Sequence of tokens that contain the relevant chunks of text

line_number¶: Position of lexer in lines as it traverses the text

advance(length: int = 1) → None¶

Moves the index forward by n characters.

Parameters:: length – The number of positions forward to move the index.

static check_for_expression_block(string: str) → bool¶: Returns True if the input string is an expression block.

consume() → str¶: Returns the current index character and advances the index 1 character.

peek() → str¶: Returns the character at the current index of the text being lexed.

peek_multiple(length: int) → str¶

Returns the next n characters from the current index in the text being lexed.

Parameters:: length – The number of characters to return

scan() → Tuple[Token, ...]¶

Tokenizes LookML into a sequence of tokens.

This method skips through the text being lexed until it finds a character that indicates the start of a new token. It consumes the relevant characters and adds the tokens to a sequence until it reaches the end of the text.

scan_comment() → CommentToken¶

Returns a token from a comment.

The initial pound (#) character is consumed in the scan method, so this method only scans for a newline or end of file to indicate the end of the token.

The pound character is added back to the beginning to the token to emphasize the importance of any leading whitespace that follows.

Example

>>> lexer = Lexer(" Disregard this line\n")
>>> lexer.scan_comment()
CommentToken(# Disregard this line)

scan_expression_block() → ExpressionBlockToken¶

Returns an token from an expression block string.

This method strips any trailing whitespace from the expression string, since Looker usually adds an extra space before the ;; terminal.

Example

>>> lexer = Lexer("SELECT * FROM ${TABLE} ;;")
>>> lexer.scan_expression_block()
ExpressionBlockToken(SELECT * FROM ${TABLE})

scan_literal() → LiteralToken¶

Returns a token from a literal string.

Example

>>> lexer = Lexer("yes")
>>> lexer.scan_literal()
LiteralToken(yes)

scan_quoted_literal() → QuotedLiteralToken¶

Returns a token from a quoted literal string.

The initial double quote character is consumed in the scan method, so this method only scans for the trailing quote to indicate the end of the token.

Example

>>> lexer = Lexer('Label"')
>>> lexer.scan_quoted_literal()
QuotedLiteralToken(Label)

scan_whitespace() → WhitespaceToken¶

Returns a token from one or more whitespace characters.

Example:

>>> lexer = Lexer("

Hello”)

>>> lexer.scan_whitespace()
LinebreakToken('

‘, 1)

lkml.parser module¶

Parses a sequence of tokenized LookML into a parse tree.

class lkml.parser.CommaSeparatedValues(_values: list = <factory>, trailing_comma: ~lkml.tree.Comma | None = None, leading_comma: ~lkml.tree.Comma | None = None)¶

Helper class to store a series of values and a flag for a trailing comma.

append(value)¶: Add a value to the private _values list.

leading_comma: Comma | None = None¶

trailing_comma: Comma | None = None¶

property values: tuple¶: Return the private _values list, cast to a tuple.

class lkml.parser.Parser(stream: Sequence[Token])¶

Parses a sequence of tokenized LookML into a parse tree.

This parser is a recursive descent parser which uses the grammar listed below (in PEG format). Each grammar rule aligns with a corresponding method (e.g. parse_expression).

Grammar:

expression ← (block / pair / list)*
block ← key literal? "{" expression "}"
pair ← key value
list ← key "[" csv? "]"
csv ← (literal / quoted_literal) ("," (literal / quoted_literal))* ","?
value ← literal / quoted_literal / expression_block
key ← literal ":"
expression_block ← [^;]* ";;"
quoted_literal ← '"' [^\"]+ '"'
literal ← [0-9A-Za-z_]+

tokens¶: A sequence of tokens to be parsed.

index¶: The position in the token sequence being parsed.

progress¶: The farthest index of progress during parsing.

depth¶: The level of recursion into nested expressions.

log_debug¶: A flag indicating that debug messages should be logged. This flag exits to turn off logging flow entirely, which provides a small performance gain compared to parsing at a non-debug logging level.

advance(length: int = 1)¶

Moves the index forward by n characters.

Parameters:: length – The number of positions forward to move the index.

check(*token_types: Type[Token], skip_trivia: bool = False) → bool¶

Compares the current index token type to specified token types.

Parameters:

*token_types – A variable number of token types to check against.
skip_trivia – Ignore trivia tokens when searching for a match.

Raises:

TypeError – If one or more of the token_types are not actual token types

Returns:

True if the current token matches one of the token_types.

consume() → Token¶: Returns the current index character and advances the index by 1 token.

consume_token_value() → str¶: Returns the value of the current index token, advancing the index 1 token.

consume_trivia(only_newlines: bool = False) → str¶: Returns all continuous trivia values.

jump_to_index(index: int)¶: Sets the parser index to a specified value.

parse() → DocumentNode¶

Main method of this class and a wrapper for the container parser.

Returns:: A document node, the root node of the LookML parse tree.

parse_block() → BlockNode | None¶

Returns a node that represents a LookML block.

Grammar: block ← key literal? "{" expression "}"

Returns:: A node with the parsed block or None if the grammar doesn’t match.

parse_comma() → Comma | None¶

parse_container() → ContainerNode¶

Returns a container node that contains any number of children.

Grammar: expression ← (block / pair / list)*

Returns:: A node with the parsed container or None if the grammar doesn’t match.
Raises:: SyntaxError – If unable to find a matching grammar rule for the stream

parse_csv() → CommaSeparatedValues | None¶

Returns a CSV object that represents comma-separated LookML values.

Returns:: A container with the parsed values or None if the grammar doesn’t match

Grammar:: csv ← ","? (literal / quoted_literal) ("," (literal / quoted_literal))* ","?

parse_key() → Tuple[SyntaxToken, Colon] | None¶

Returns a syntax token that represents a literal key and colon character.

Grammar: key ← literal ":"

Returns:: A tuple of syntax tokens with the parsed key and colon or None if the grammar doesn’t match.

parse_list() → ListNode | None¶

Returns a node that represents a LookML list.

Grammar: list ← key "[" csv? "]"

Returns:: A node with the parsed list or None if the grammar doesn’t match

parse_pair() → PairNode | None¶

Returns a dictionary that represents a LookML key/value pair.

Grammar: pair ← key value

Returns:: A dictionary with the parsed pair or None if the grammar doesn’t match.

parse_value(parse_prefix: bool = False, parse_suffix: bool = False) → SyntaxToken | None¶

Returns a syntax token that represents a value.

Grammar: value ← literal / quoted_literal / expression_block

Returns:: A syntax token with the parsed value or None if the grammar doesn’t match.

peek() → Token¶: Returns the token at the current index.

lkml.parser.backtrack_if_none(fn)¶

Decorates parsing methods to backtrack to a previous position on failure.

This method sets a marker at the current position before attempting to run a parsing method. If the parsing method fails and returns None, it resets the index to the marker.

It also keeps track of the farthest index of progress in case all parsing methods fail and we need to return a SyntaxError to the user with a character number.

Parameters:: fn (Callable) – The method to be decorated for backtracking.

lkml.simple module¶

Interface classes between the parse tree and a data structure of primitives.

These classes facilitate parsing and generation to and from simple data structures like lists and dictionaries, and allow users to parse and generate LookML without needing to interact with the parse tree.

class lkml.simple.DictParser¶

Parses a Python dictionary into a parse tree.

Review the grammar specified for the Parser class to understand how LookML is represented. The grammar details the differences between blocks, pairs, keys, and values.

parent_key¶: The name of the key at the previous level in a LookML block.

level¶: The number of indentations appropriate for the current position.

base_indent¶: Whitespace representing one tab.

latest_node¶: The type of the last node to be parsed.

decrease_level() → None¶: Decreases the indent level of the current line by one tab.

expand_list(key: str, values: Sequence) → List[BlockNode | ListNode | PairNode]¶

Expands and parses a list of values for a repeatable key.

Parameters:

key – A repeatable LookML field type (e.g. “views” or “dimension_groups”)
values – A sequence of objects to be parsed

Returns:

A list of block, list, or pair nodes, depending on the list’s contents.

increase_level() → None¶

Increases the indent level of the current line by one tab.

This also resets the latest node, mainly for formatting reasons.

property indent: str¶: Returns the level-adjusted indent.

is_plural_key(key: str) → bool¶

Returns True if the key is a repeatable key.

For example, dimension can be repeated, but sql cannot be.

The key allowed_value is a special case and changes behavior depending on its parent key. If its parent key is access_grant, it is a list and cannot be repeated. Otherwise, it can be repeated.

The parent key query is also a special case, where children are kept as lists. See issue #53.

Parameters:: key – The name of the key to test.

property newline_indent: str¶: Returns a newline plus the current indent.

parse(obj: Dict[str, Any]) → DocumentNode¶: Parses a primitive representation of LookML into a parse tree.

Dynamically serializes a Python object based on its type.

Parameters:

key – A LookML field type (e.g. “suggestions” or “hidden”)
value – A string, tuple, or list to serialize

Raises:

TypeError – If input value is not of a valid type

Returns: