Динамично Созданные Регулярные выражения работают чрезвычайно медленный!

Having a potential of 50 match groups in a single expression by default is going to be a bit slow. I would do a few things to see if you can pin down the performance setback.

1. Start by trying a hard coded, vs dynamic comparison and see if you have any performance benefit.
2. Look at your requirements and see if there is any way you can reduce the number of groupings that you need to evaluate
3. Use a profiler tool if needed, such as Ants Profiler to see the location of the slowdown.

Well. Building the pattern using a StringBuilder will save a few cycles, compared to concatenating them.

An optimization on this that is drastic (can be visibly measured) is most likely going to be doing this through some other method.

Regular expressions are slow ... powerful but slow. Parsing through a text-file and then comparing using regular expressions just to retrieve the right bits of data is not going to be very quick (dependent on the host computer and size of text file).

Perhaps storing the data in some other method rather than a large text file would be more efficient to parse (use XML for that as well?), or perhaps a comma separated list.

I would just build a lexer by hand.

In this case it looks like you have a bunch of fields seperated by tabs, with a record terminated by a new line. The XML file appears to describe the sequence of columns, and their types.

Writing code to recognize each case by hand is probably 5-10 lines of code at the worst case.

You would then want to simply generate an arraay of PrimitiveType[] values from the xml file, and then call the "GetValues" function below.

This should allow you to make a single pass through the input stream, which should give a big boost over using regexes.

You'll need to supply the "ScanXYZ" methods your self. They should be easy to write. It's best to implement them w/out using regexes.

public IEnumerable<object[]> GetValues(TextReader reader, PrimitiveType[] schema)
{
   while (reader.Peek() > 0)
   {
       var values = new object[schema.Length];
       for (int i = 0; i < schema.Length; ++i)
       {
           switch (schema[i])
           {
               case PrimitiveType.BIT:
                   values[i] = ScanBit(reader);
                   break;
               case PrimitiveType.DATE:
                   values[i] = ScanDate(reader);
                   break;
               case PrimitiveType.FLOAT:
                   values[i] = ScanFloat(reader);
                   break;
               case PrimitiveType.INTEGER:
                   values[i] = ScanInt(reader);
                   break;
               case PrimitiveType.STRING:
                   values[i] = ScanString(reader);
                   break;
           }
       }

       EatTabs(reader);

       if (reader.Peek() == '\n')
       {
            break;
       }

   if (reader.Peek() == '\n')
   {
       reader.Read();
   }
   else if (reader.Peek() >= 0)
   {
       throw new Exception("Extra junk detected!");
   }

   yield return values;

   }

   reader.Read();
}

Some performance thoughts:

• use [01] instead of (0|1)
• use non-capturing groups (?:expr) instead of capturing groups (if you really need grouping)

Edit As it seems that your values are separated by whitespace, why don’t you split it up there?

Regular expression are expensive to create and are even more expensive if you compile them. So the problem is that you are creating many regular expressions but use them only once.

You should cache them for reuse and relly don't compile them if you don't want to use them really often. I have never meassured that, but I could imagine that you will have to use a simple regular expression well over 100 times to outweight the cost of the compilation.

Performance test

• Regex: "^(?:[a-zA-Z0-9](?:[a-zA-Z0-9-]*[a-zA-Z0-9])?\.)+(?:[a-z]{2}|com|org|net|gov|mil|biz|info|mobi|name|aero|jobs|museum)$"

• Input: "www.stackoverflow.com"

• Results in milliseconds per iteration

  • one regex, compiled, 10,000 iterations: 0.0018 ms
  • one regex, not compiled, 10,000 iterations: 0.0021 ms
  • one regex per iteration, not compiled, 10,000 iterations: 0.0287 ms
  • one regex per iteration, compiled, 10,000 iterations: 4.8144 ms

Note that even after 10,000 iterations the compiled and uncompiled regex are still very close together comparing their performance. With increasing number of iterations the compiled regex performs better.

• one regex, compiled, 1,000,000 iterations: 0.00137 ms
• one regex, not compiled, 1,000,000 iterations: 0.00225 ms