# Abstract Class yii\db\SqlTokenizer

Inheritance | yii\db\SqlTokenizer » yii\base\Component » yii\base\BaseObject
Implements | yii\base\Configurable
Subclasses | yii\db\sqlite\SqlTokenizer
Available since version | 2.0.13
Source Code | https://github.com/yiisoft/yii2/blob/master/framework/db/SqlTokenizer.php

SqlTokenizer splits an SQL query into individual SQL tokens.

It can be used to obtain an addition information from an SQL code.

Usage example:

    $tokenizer = new SqlTokenizer("SELECT * FROM user WHERE id = 1");
    $root = $tokeinzer->tokenize();
    $sqlTokens = $root->getChildren();

Tokens are instances of yii\db\SqlToken.

## Public Properties

Hide inherited properties

Property | Type | Description | Defined By
--- | --- | --- | ---
$behaviors | yii\base\Behavior[] | List of behaviors attached to this component. This property is read-only. | yii\base\Component
$sql | string | SQL code. | yii\db\SqlTokenizer

## Protected Properties

Hide inherited properties

Property | Type | Description | Defined By
--- | --- | --- | ---
$length | integer | SQL code string length. | yii\db\SqlTokenizer
$offset | integer | SQL code string current offset. | yii\db\SqlTokenizer

## Public Methods

Hide inherited methods

Method | Description | Defined By
--- | --- | ---
__call() | Calls the named method which is not a class method. | yii\base\Component
__clone() | This method is called after the object is created by cloning an existing one. | yii\base\Component
__construct() | Constructor. | yii\db\SqlTokenizer
__get() | Returns the value of a component property. | yii\base\Component
__isset() | Checks if a property is set, i.e. defined and not null. | yii\base\Component
__set() | Sets the value of a component property. | yii\base\Component
__unset() | Sets a component property to be null. | yii\base\Component
attachBehavior() | Attaches a behavior to this component. | yii\base\Component
attachBehaviors() | Attaches a list of behaviors to the component. | yii\base\Component
behaviors() | Returns a list of behaviors that this component should behave as. | yii\base\Component
canGetProperty() | Returns a value indicating whether a property can be read. | yii\base\Component
canSetProperty() | Returns a value indicating whether a property can be set. | yii\base\Component
className() | Returns the fully qualified name of this class. | yii\base\BaseObject
detachBehavior() | Detaches a behavior from the component. | yii\base\Component
detachBehaviors() | Detaches all behaviors from the component. | yii\base\Component
ensureBehaviors() | Makes sure that the behaviors declared in behaviors() are attached to this component. | yii\base\Component
getBehavior() | Returns the named behavior object. | yii\base\Component
getBehaviors() | Returns all behaviors attached to this component. | yii\base\Component
hasEventHandlers() | Returns a value indicating whether there is any handler attached to the named event. | yii\base\Component
hasMethod() | Returns a value indicating whether a method is defined. | yii\base\Component
hasProperty() | Returns a value indicating whether a property is defined for this component. | yii\base\Component
init() | Initializes the object. | yii\base\BaseObject
off() | Detaches an existing event handler from this component. | yii\base\Component
on() | Attaches an event handler to an event. | yii\base\Component
tokenize() | Tokenizes and returns a code type token. | yii\db\SqlTokenizer
trigger() | Triggers an event. | yii\base\Component

## Protected Methods

Hide inherited methods

Method | Description | Defined By
--- | --- | ---
indexAfter() | Returns an index after the given string in the SQL code starting with the specified offset. | yii\db\SqlTokenizer
isComment() | Returns whether there's a commentary at the current offset. | yii\db\SqlTokenizer
isIdentifier() | Returns whether there's an identifier at the current offset. | yii\db\SqlTokenizer
isKeyword() | Returns whether the given string is a keyword. | yii\db\SqlTokenizer
isOperator() | Returns whether there's an operator at the current offset. | yii\db\SqlTokenizer
isStringLiteral() | Returns whether there's a string literal at the current offset. | yii\db\SqlTokenizer
isWhitespace() | Returns whether there's a whitespace at the current offset. | yii\db\SqlTokenizer
startsWithAnyLongest() | Returns whether the longest common prefix equals to the SQL code of the same length at the current offset. | yii\db\SqlTokenizer
substring() | Returns a string of the given length starting with the specified offset. | yii\db\SqlTokenizer

## Property Details

$length protected property

SQL code string length.

protected integer $length = null
$offset protected property

SQL code string current offset.

protected integer $offset = null
$sql public property

SQL code.

public string $sql = null

## Method Details

__construct() public method

Constructor.

public void __construct ( $sql, $config = [] )
$sql | string | SQL code to be tokenized.
$config | array | Name-value pairs that will be used to initialize the object properties

indexAfter() protected method

Returns an index after the given string in the SQL code starting with the specified offset.

protected integer indexAfter ( $string, $offset = null )
$string | string | String to be found.
$offset | integer|null | SQL code offset, defaults to current if `null` is passed.
return | integer | Index after the given string or end of string index.

isComment() protected abstract method

Returns whether there's a commentary at the current offset.

If this methos returns `true`, it has to set the `$length` parameter to the length of the matched string.

protected abstract boolean isComment ( &$length )
$length | integer | Length of the matched string.
return | boolean | Whether there's a commentary at the current offset.

isIdentifier() protected abstract method

Returns whether there's an identifier at the current offset.

If this methos returns `true`, it has to set the `$length` parameter to the length of the matched string. It may also set `$content` to a string that will be used as a token content.

protected abstract boolean isIdentifier ( &$length, &$content )
$length | integer | Length of the matched string.
$content | string | Optional content instead of the matched string.
return | boolean | Whether there's an identifier at the current offset.

isKeyword() protected abstract method

Returns whether the given string is a keyword.

The method may set `$content` to a string that will be used as a token content.

protected abstract boolean isKeyword ( $string, &$content )
$string | string | String to be matched.
$content | string | Optional content instead of the matched string.
return | boolean | Whether the given string is a keyword.

isOperator() protected abstract method

Returns whether there's an operator at the current offset.

If this methos returns `true`, it has to set the `$length` parameter to the length of the matched string. It may also set `$content` to a string that will be used as a token content.

protected abstract boolean isOperator ( &$length, &$content )
$length | integer | Length of the matched string.
$content | string | Optional content instead of the matched string.
return | boolean | Whether there's an operator at the current offset.

isStringLiteral() protected abstract method

Returns whether there's a string literal at the current offset.

If this methos returns `true`, it has to set the `$length` parameter to the length of the matched string. It may also set `$content` to a string that will be used as a token content.

protected abstract boolean isStringLiteral ( &$length, &$content )
$length | integer | Length of the matched string.
$content | string | Optional content instead of the matched string.
return | boolean | Whether there's a string literal at the current offset.

isWhitespace() protected abstract method

Returns whether there's a whitespace at the current offset.

If this methos returns `true`, it has to set the `$length` parameter to the length of the matched string.

protected abstract boolean isWhitespace ( &$length )
$length | integer | Length of the matched string.
return | boolean | Whether there's a whitespace at the current offset.

startsWithAnyLongest() protected method

Returns whether the longest common prefix equals to the SQL code of the same length at the current offset.

protected boolean startsWithAnyLongest ( array &$with, $caseSensitive, &$length = null, &$content = null )
$with | string[] | Strings to be tested. The method will modify this parameter to speed up lookups.
$caseSensitive | boolean | Whether to perform a case sensitive comparison.
$length | integer|null | Length of the matched string.
$content | string|null | Matched string.
return | boolean | Whether a match is found.

substring() protected method

Returns a string of the given length starting with the specified offset.

protected string substring ( $length, $caseSensitive = true, $offset = null )
$length | integer | String length to be returned.
$caseSensitive | boolean | If it's `false`, the string will be uppercased.
$offset | integer|null | SQL code offset, defaults to current if `null` is passed.
return | string | Result string, it may be empty if there's nothing to return.

tokenize() public method

Tokenizes and returns a code type token.

public yii\db\SqlToken tokenize ( )
return | yii\db\SqlToken | Code type token.