sqliteparser

SQLite Parser

A lightweight Go parser for SQLite SQL statements that properly handles SQLite-specific unquoted literals like NULL, integers, and booleans. This parser was designed specifically to address the issue of parsing SQL statements with unquoted NULL values.

Features

• Lexer that tokenizes SQLite SQL statements with SQLite-specific syntax
• AST (Abstract Syntax Tree) representation of SQL literals and expressions
• Simple parser that extracts literals from SQL statements
• Special handling for SQLite's unquoted literals:
  • NULL literals
  • Integer literals (1, 42, etc.)
  • Boolean literals (TRUE, FALSE)
  • BLOB literals (X'HEXSTRING')
• Parsing of UPDATE statement SET pairs
• Parsing of WHERE conditions, including IS NULL and IS NOT NULL

Installation

go get github.com/Alibi85/sqliteparser

Usage

import (
    "fmt"
    "github.com/Alibi85/sqliteparser"
    "github.com/Alibi85/sqliteparser/ast"
)

func main() {
    // Example SQL statement with unquoted NULL
    sql := "UPDATE users SET is_admin = NULL WHERE id = 42"
    
    // Extract all literals from the SQL
    literals := sqliteparser.ExtractLiterals(sql)
    for _, lit := range literals {
        fmt.Printf("Literal: %T %s\n", lit, lit.String())
    }
    
    // Parse UPDATE statement SET pairs
    pairs := sqliteparser.ParseUpdateSetPairs(sql)
    for _, pair := range pairs {
        fmt.Printf("Set: %s = %s\n", pair.Column, pair.Value)
        
        // Check if the value is NULL
        if sqliteparser.IsNull(pair.Value) {
            fmt.Printf("The value for %s is NULL\n", pair.Column)
        }
    }
    
    // Parse WHERE condition
    condition := sqliteparser.ParseCondition(sql)
    if condition != nil {
        fmt.Printf("Condition: %s\n", condition.String())
    }
}

Example Tool

The package includes a command-line tool in the examples directory:

go run examples/main.go 'UPDATE users SET is_admin = NULL WHERE id = 42'

This will display:

Parsing SQL: UPDATE users SET is_admin = NULL WHERE id = 42
-----------------------------------------
Extracting literals:
  1: *ast.NullLiteral: NULL
  2: *ast.IntegerLiteral: 42

Parsing UPDATE set pairs:
  1: is_admin = NULL

Parsing WHERE condition:
  Condition: (id = 42)

Token stream:
  UPDATE: 'UPDATE'
  IDENT: 'users'
  SET: 'SET'
  IDENT: 'is_admin'
  EQ: '='
  NULL: 'NULL'
  WHERE: 'WHERE'
  IDENT: 'id'
  EQ: '='
  INTEGER: '42'
  EOF: ''

Components

The parser is composed of three main components:

1. Lexer (lexer package): Tokenizes SQL input into a stream of tokens
2. AST (ast package): Defines the structure of the parse tree
3. Parser (parser package): Extracts literals and expressions from SQL statements

Primary Use Case

This parser was created to address the specific challenge of parsing SQL statements with unquoted NULL values, which caused issues in previous SQL parsers. It's particularly useful for:

1. Correctly identifying NULL literals in SQL statements
2. Parsing SET clauses in UPDATE statements
3. Handling WHERE conditions with NULL values (both = NULL and IS NULL forms)

License

This code is licensed under the MIT License. See the LICENSE file for details.