Since we just released a CTP of PowerShell V2, I thought I’d share a handy little script to demonstrate one of the new APIs we introduced: Show-ColorizedContent.ps1.

This CTP introduces a new tokenizer API that lets you work with PowerShell script content the same way that our parser does – as a collection of items (tokens) that represent the underlying structure of that script Until now, any tool that works with the data of a PowerShell script needs to parse the script on its own – usually with fragile regular expressions or other means.

This often works, but usually falls apart on complex scripts:

In the first line, “Write-Host” is an argument to the Write-Host cmdlet, but gets parsed as a string. Fair enough, but the second line does not treat the argument the same way. In fact, since it matches a cmdlet name, the argument gets parsed as another cmdlet call. In the here string that follows, the Write-Host cmdlet name gets highlighted again, even though it is really just part of a string.

This is absolutely not a slam on the authors of existing highlighters – it’s just that we’ve now introduced something that makes life so much easier.

$content = [IO.File]::ReadAllText("c:\temp\ContentTest.ps1")
$errors = [System.Management.Automation.PSParseError[]] @()
[System.Management.Automation.PsParser]::Tokenize($content, [ref] $errors)

This API generates a collection of PSToken objects that give all the information you need to properly dissect a PowerShell script:

PS C:\\Temp> \[System.Management.Automation.PsParser\]::Tokenize($content, \[ref\] $errors) | ft -auto

Content                           Type Start Length StartLine StartColumn EndLine EndColumn
-------                           ---- ----- ------ --------- ----------- ------- ---------
Write-Host                     Command     0     10         1           1       1        11
Write-Host                      String    11     12         1          12       1        24
...                            NewLine    23      2         1          24       2         1
Write-Host                     Command    25     10         2           1       2        11
Write-Host             CommandArgument    36     10         2          12       2        22
...                            NewLine    46      2         2          22       3         1
...                            NewLine    48      2         3           1       4         1
Write-Host Write-Host           String    50     23         4           1       4        24
...                            NewLine    73      2         4          24       5         1
...                            NewLine    75      2         5           1       6         1
testContent                   Variable    77     12         6           1       6        13
=                             Operator    90      1         6          14       6        15
Write-Host Hello World          String    92     30         6          16       8         3
...                            NewLine   122      2         8           3       9         1

This adds a whole new dimension to the way you can interact with PowerShell. Some natural outcomes are:

• syntax highlighting
• preparing a script for production (replacing all aliased commands with their expanded equivalent, etc)
• script refactoring
• FxCop / Style guideline checks
• PowerTab :)

As a starter example, I’ve attached Show-ColorizedContent.ps1 – a script to colorize PowerShell scripts in a console window. Its primary goal is to support demonstrations of PowerShell snippets. For that, it adds line numbers to let you easily refer to portions of your script. It also includes a -HighlightRanges parameter to let you highlight specific ranges of the script. The -HighlightRanges parameter is an array of line numbers, which you can easily create using PowerShell’s standard array range syntax:

Enjoy – you can download it here.