Following my experiment to write a parser for Visual Studio solution files using the Sprache library, I’d like to share a few Sprache techniques that I found useful.

Parse a token into an enum value

The Visual Studio solution file format includes a set of Project definitions, each with one or more ProjectSection definitions, as well as a collection of GlobalSection definitions.

    GlobalSection(SolutionConfigurationPlatforms) = preSolution
        Debug|Any CPU = Debug|Any CPU
        Release|Any CPU = Release|Any CPU
    EndGlobalSection

Each ProjectSection and GlobalSection contains a token (preSolution in this case) that instructs Visual Studio when it’s contents are required during the process of opening a solution file:

To represent this in my data model I created the following enum to express these loading sequence tokens:

    public enum SectionLoadSequence
    {
        Unrecognized,
        PreSolution,
        PostSolution,
        PreProject,
        PostProject
    }

The Pre- and Post- nodes should be self-explanitory, but I want to temporarily call attention to the Unrecognized value.

I have found it useful to introduce in my grammars the concept of an unrecognized section which, while it may not survive into the final draft of a particular grammer, has allowed my parsers to handle structured content for which I have not yet written a detailed parser. In short, it has helped me during development, and whether it survives into the final draft becomes a question of how you want your parser to respond to input that is either not well-formed, or for which the format has changed.

But I digress.

Two of the fun constructs that ship with the Sprache library are the concept of Or, which lets you link alternative elements together, and Return, which lets you substitute anything you want in place of the parsed input.

Using these constructs, it becomes straightforward to write a parser for a range of enum values:

    public static readonly Parser LoadSequence =
        from sequence in Parse.String("preSolution").Token().Return(SectionLoadSequence.PreSolution)
                     .Or(Parse.String("postSolution").Token().Return(SectionLoadSequence.PostSolution))
                     .Or(Parse.String("preProject").Token().Return(SectionLoadSequence.PreProject))
                     .Or(Parse.String("postProject").Token().Return(SectionLoadSequence.PostProject))
                                                    .Or(Parse.Return(SectionLoadSequence.Unrecognized))
        select sequence;

Very expressive.

Parse unique inner content based on an opening token

Another useful construct that ships with Sprache is Then, which lets you determine which expectation follows based on the contents matched in the previous expression.

This is useful in the case of solution files because there are different types of GlobalSection, each with their own inner format.

Take, for example, the difference between a SolutionProperties global section:

    GlobalSection(SolutionProperties) = preSolution
        HideSolutionNode = FALSE
    EndGlobalSection

and a ProjectConfigurationPlatforms global section:

    GlobalSection(ProjectConfigurationPlatforms) = postSolution
        {AE7D2A46-3F67-4986-B04B-7DCE79A549A5}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
        {AE7D2A46-3F67-4986-B04B-7DCE79A549A5}.Debug|Any CPU.Build.0 = Debug|Any CPU
        {AE7D2A46-3F67-4986-B04B-7DCE79A549A5}.Release|Any CPU.ActiveCfg = Release|Any CPU
        {AE7D2A46-3F67-4986-B04B-7DCE79A549A5}.Release|Any CPU.Build.0 = Release|Any CPU
    EndGlobalSection

Parsing (and eventually visualizing or manipulating) this last section’s inner content was my whole motivation for parsing solution files in the first place, but looking at the two together it is clear that the parser required to extract relevant details from the ProjectConfigurationPlatforms section would fail to parse a SolutionProperties section.

Luckily, the Then construct takes a lambda accepting the parsed content as an argument, so you can do something funky like this:

    public static readonly Parser GlobalSection =
        from start in Parse.String("GlobalSection").Token()
        from section in RoundBracketedString.Then(s => 
            s == "SolutionProperties" ? SolutionPropertiesGlobalSection
            : s == "SolutionConfigurationPlatforms" ? SolutionConfigurationPlatformsGlobalSection
            : s == "ProjectConfigurationPlatforms" ? ProjectConfigurationPlatformsGlobalSection
                                                   : UnrecognizedGlobalSection(s))
        from end in Parse.String("EndGlobalSection").Token()
        select section;

In between the GlobalSection and EndGlobalSection tags, we first parse the RoundBracketedString that differentiates what kind of global section we’re dealing with. Then we accept the inner content of that round bracketed string and supply different parsers customized for the expected format of each type of section.

Notice again that I have created an UnrecognizedGlobalSection that accepts the section type as an argument. This parser simply swallows everything until the next EndGlobalSection tag, saving it for later use in a diagnostic message while allowing the parsing to continue without exception.

Tip of the iceberg

These two use cases represent just the tip of the iceberg in terms of the possibliies offered by Sprache.

I’m very excited by this library and look forward to using it to explore even more complex grammars.

What content have you had to parse that could benefit from this level of expressiveness?

Anyone who has worked with Visual Studio and more than one branch of active development has been bitten by the friction of merging solution files.

The format of solution files is a bit esoteric, with lots of key=value pairs and guids representing projects, project types, and build configurations. Merging individual lines often requires so much effort that most devs I’ve spoken wtih recommend picking one branch or the other and manually re-constructing the other branch’s changes. Needless to say this work is fidly and error-prone.

Somewhere I got it into my head that building a tool to help this would be a good idea, and apparently I’m not alone in that thought. When sitting down to write this post, I discovered the SLNTools project on codeplex; it looks interesting and I’ll have to try it out.

In the meantime, however, I wanted to share with you a bit about my experience building a parser for solution files using the Sprache library developed by Nicholas Blumhardt.

Some background

The idea behind Sprache is to fill the void between Regular Expression for parsing simple things and using full-blown DSL or parsing toolkits to describe complete grammars on the other.

Sprache takes a functional or parser combinator [PDF] approach; the library provides a set of .NET classes and extension methods that handle low-level parsing jobs like consuming input one character at a time and defining sequence and look-ahead expectations. These tools are exposed as a collection of Parser<T> constructs (where T is the type returned by a successful parse) that can be composed in a declarative style using LINQ query comprehension syntax.

The resulting parser can be very expressive, readable, and testable.

I’ll share some examples as I build up my SolutionFileGrammar below, but here’s a taste to whet your whistle:

public static readonly Parser<SolutionFile> Solution =
        from header in Header
        from projects in Project.Many().Optional()
        from globals in Global
        select new SolutionFile(header, projects, globals);

About Solution Files

In order to define a parser for the Solution File format we have to understand how it is structured.

The best description of solution file syntax I have found is this excerpt: Hack the Project and Solution Files.

Let’s take a look at a empty VS2012 solution file:

Microsoft Visual Studio Solution File, Format Version 12.00
# Visual Studio 2012
Global
    GlobalSection(SolutionProperties) = preSolution
        HideSolutionNode = FALSE
    EndGlobalSection
EndGlobal

And now one with a single project and it’s default build configurations:

Microsoft Visual Studio Solution File, Format Version 12.00
# Visual Studio 2012
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "HttpWebAdapters", "HttpWebAdapters\HttpWebAdapters.csproj", "{AE7D2A46-3F67-4986-B04B-7DCE79A549A5}"
EndProject
Global
    GlobalSection(SolutionConfigurationPlatforms) = preSolution
        Debug|Any CPU = Debug|Any CPU
        Release|Any CPU = Release|Any CPU
    EndGlobalSection
    GlobalSection(ProjectConfigurationPlatforms) = postSolution
        {AE7D2A46-3F67-4986-B04B-7DCE79A549A5}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
        {AE7D2A46-3F67-4986-B04B-7DCE79A549A5}.Debug|Any CPU.Build.0 = Debug|Any CPU
        {AE7D2A46-3F67-4986-B04B-7DCE79A549A5}.Release|Any CPU.ActiveCfg = Release|Any CPU
        {AE7D2A46-3F67-4986-B04B-7DCE79A549A5}.Release|Any CPU.Build.0 = Release|Any CPU
    EndGlobalSection
    GlobalSection(SolutionProperties) = preSolution
        HideSolutionNode = FALSE
    EndGlobalSection
EndGlobal

And finally, one with Nuget Package Restore enabled, which adds a solution folder and some freestanding solution items:

Microsoft Visual Studio Solution File, Format Version 12.00
# Visual Studio 2012
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "HttpWebAdapters", "HttpWebAdapters\HttpWebAdapters.csproj", "{AE7D2A46-3F67-4986-B04B-7DCE79A549A5}"
EndProject
Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = ".nuget", ".nuget", "{8374A24A-6031-48CB-8B66-A2B510FA251F}"
ProjectSection(SolutionItems) = preProject
        .nuget\NuGet.Config = .nuget\NuGet.Config
        .nuget\NuGet.exe = .nuget\NuGet.exe
        .nuget\NuGet.targets = .nuget\NuGet.targets
    EndProjectSection
EndProject
Global
    GlobalSection(SolutionConfigurationPlatforms) = preSolution
        Debug|Any CPU = Debug|Any CPU
        Release|Any CPU = Release|Any CPU
    EndGlobalSection
    GlobalSection(ProjectConfigurationPlatforms) = postSolution
        {AE7D2A46-3F67-4986-B04B-7DCE79A549A5}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
        {AE7D2A46-3F67-4986-B04B-7DCE79A549A5}.Debug|Any CPU.Build.0 = Debug|Any CPU
        {AE7D2A46-3F67-4986-B04B-7DCE79A549A5}.Release|Any CPU.ActiveCfg = Release|Any CPU
        {AE7D2A46-3F67-4986-B04B-7DCE79A549A5}.Release|Any CPU.Build.0 = Release|Any CPU
    EndGlobalSection
    GlobalSection(SolutionProperties) = preSolution
        HideSolutionNode = FALSE
    EndGlobalSection
EndGlobal

And now, the parser

In order to parse this with Sprache we need to decompose it into smaller and smaller pieces, build parsers for each of those pieces, then assemble those pieces into a grammar.

We experess that Grammar with a series of static methods that can be built-up test-first by starting with one of the smallest or inner-most pieces of nested content.

For eample, let’s take the header first:

Microsoft Visual Studio Solution File, Format Version 12.00
# Visual Studio 2012

The relevant pieces of information are: - a version declaration; and - a product description

Taking that first part, the version declaration, we can focus smaller still on just the version number itself:

[Test]
public void SolutionVersionNumber_is_Number_period_Number()
{
    var input = @"12.00";

    var result = SolutionFileGrammar.SolutionVersionNumber.Parse(input);

    Assert.AreEqual(12, result.Major);
    Assert.AreEqual(00, result.Minor);
}

And write a parser for it:

    public static readonly Parser<SolutionVersionNumber> SolutionVersionNumber =
        from rawMajor in Parse.Number.Token()
        from period in Parse.Char('.')
        from rawMinor in Parse.Number.Token()
        let major = int.Parse(rawMajor)
        let minor = int.Parse(rawMinor)
        select new SolutionVersionNumber(major, minor);

There are a number of things going on here so let’s break it down.

First off we declare a Parser<SolutionVersionNumber> that will return our parsed result and then declare that a SolutionVersionNumber is composed of:

• A number…

  from rawMajor in Parse.Number.Token()
  

• …followed by a period…

  from period in Parse.Char('.')
  

• …followed by a number

  from rawMinor in Parse.Number.Token()
  

• then I use the let keyword to transform that parsed text into integers

  let major = int.Parse(rawMajor)
  let minor = int.Parse(rawMinor)
  

• and finally the select keyword to create a new instance of my result:

  select new SolutionVersionNumber(major, minor);
  

Where a SolutionFileVersionNumber is part of our data model:

public class SolutionVersionNumber
{
    public SolutionVersionNumber(int major, int minor)
    {
        Major = major;
        Minor = minor;
    }

    public int Major { get; private set; }
    public int Minor { get; private set; }
}

then came the product name

Now let’s focus on the second piece that we want to parse, the product name:

# Visual Studio 2012

with a test:

[Test]
public void ProductName_is_pound_followed_by_text()
{
    var input = @"# Visual Studio 2012";

    var result = SolutionFileGrammar.ProductName.Parse(input);

    Assert.AreEqual("Visual Studio 2012", result);
}

and a parser:

public static readonly Parser<string> ProductName =
    from pound in Parse.Char('#').Token()
    from name in Parse.AnyChar.Until(NewLine.Or(Eof)).Text()
    select name;

and now the fun begins

Now that we have a simple, tested parser for the version number and another one for the product name, let’s write a test and parser for the whole header:

Given:

Microsoft Visual Studio Solution File, Format Version 12.00
# Visual Studio 2012

here’s the test:

    [Test]
    public void Header_contains_version_information()
    {
        var input =
@"Microsoft Visual Studio Solution File, Format Version 12.00
# Visual Studio 2012";

        var result = SolutionFileGrammar.Header.Parse(input);

        Assert.AreEqual("Visual Studio 2012", result.ProductName);
        Assert.AreEqual(12, result.MajorVersion);
        Assert.AreEqual(00, result.MinorVersion);
    }

the parser:

public static readonly Parser<SolutionFileHeader> Header =
    from ignore1 in Parse.String("Microsoft Visual Studio Solution File, Format Version").Token()
    from version in SolutionVersionNumber
    from name in ProductName
    select new SolutionFileHeader(version, name);

and the object model:

public class SolutionFileHeader
{
    public SolutionFileHeader(SolutionVersionNumber version, string productName)
    {
        MajorVersion = version.Major;
        MinorVersion = version.Minor;
        ProductName = productName;
    }

    public int MajorVersion { get; private set; }
    public int MinorVersion { get; private set; }
    public string ProductName { get; private set; }
}

in conclusion

With that I hope that you can see how expressive a Sprache-based grammar is. It takes a bit of discipline to start with an individual token and grow your grammar test-first, but as your library of tokens grows, the size of your parsers grows also, and before long the declarative and combinatorial nature of Sprache has you moving along at a fast clip.

Full source code for this article, and a more comprehensive (though not yet complete) grammar, is available on github: https://github.com/davidalpert/viper