The “For in” style of enumeration in Delphi was introduced in Delphi 2005. The idea is to make enumeration over a collection of things easier and cleaner by removing the need for the iterator variable, removing the code for indexing and removing the code for boundary checking. If you think you know everything about enumeration, hold on to your hat. You may be in for a pleasant surprise.
We will start with the basics and then progress to custom enumerators. Seasoned “enumerophiles” may prefer to skip to the “Custom Enumeration over Objects” section. Let’s look at a simple example. Say we’ve got a list of strings, and we want to show them to the user. Here is how we would do it before and after this feature.
Enumeration by index
procedure EnumDemo1; var SomeStrings: array of string; s: string; i: integer; begin Populate( SomeStrings); for i := 0 to Length( SomeStrings) - 1 do begin s := SomeStrings[i]; Dialogs.ShowMessage( s) end end;
Enumeration by for-in
procedure EnumDemo2; var SomeStrings: array of string; s: string; begin Populate( SomeStrings); for s in SomeStrings do Dialogs.ShowMessage( s); end;
Looking at the details of the above, you see that for-in enumeration is not exactly earth-shattering. We have dropped a variable declaration, the for-loop is a little cleaner and indexing is compiler-magiced away. But you’ve got the wrong idea if you think that the key selling point of for-in enumeration is these simple syntactic abbreviations. The real selling point is greater than that.
Stand back from the details of the code, and think about what we are doing at an abstract level. We are looping through a collection of strings and doing something with the members. From this perspective, the mechanism of indexing is a pesky irrelevant detail. What manner of collection it is (array, list, something-else) is a pesky irrelevant detail. Boundary checking? – another irrelevant artefact of implementation. What we have achieved in EnumDemo2() over EnumDemo1(), is the holy grail of language design – a closer alignment of semantics with syntax.
So what can we enumerate over, out-of-the-box? The following VCL types are for-in enumerable? Member types are as you would expect.
Fundamental and compound types – Enumerables
- Dynamic arrays
- Static arrays
- string/ ansistring/ utf8string
Sets are a special case. Enumeration only visits members that are in the set, as opposed to the base range.
- A large suite of RTTI properties.
Interface pointer Enumerables
So what is missing from Delphi out-of-the-box for-in enumeration support?
Perhaps in a future version of Delphi we will get enumeration support for:
- Enumeration types. With some custom declarations, enumeration of “enumerated types” has been solved by Jeroen W. Pluimers
- Set inversion. Currently, there is no way to for-in enumerate over members NOT in set. Wouldn’t it be handy if we had a generic function to invert sets.
- Character Streams. Ever wanted to parse a text file character by character? Wouldn’t it be nice if we could use for-in?
- Reverse order. Have you ever wanted to traverse in reverse order? How useful would this be?
- TDataset – How many times have you written a while-not-eof loop? For us older developers, the number is probably astronomical.
- Predicates. Ever wanted to traverse a collection, but only for those members that met some specified condition. Sure we can do this with a simple if statement, but this pattern is so common, the thought of an XSLT style syntax mechanism for predicates is enticing. Imagine if we could do something like FantasyEnum4()
Predicated enumeration as it is now…
procedure EnumDemo3( Collection: TObjectList<TSomeClass>); var Member: TSomeClass; begin for Member in Collection do if SomeCondition( Member) then Member.SomeAction end;
Fantasy enumeration supporting predicates
procedure FantasyEnum4( Collection: TObjectList<TSomeClass>); var Member: TSomeClass; begin for Member in Collection ! SomeCondition( Member) do Member.SomeAction end;
Generics and Enumeration
Generic classes and for-in enumeration are a good match-up. Take a look at the syntactic synergy in EnumDemo3 above. If we wanted to do the same in Delphi 7, we would probably write something like EnumDemo5() below…
procedure EnumDemo5( Collection: TObjectList); var i: integer; Member: TSomeClass; begin for i := 0 to Collection.Count - 1 do begin Member := Collection[i] as TSomeClass; if SomeCondition( Member) then Member.SomeAction end end;
Custom Enumeration over objects
So much for the basics. Now here is where it gets really interesting. With some customisation you can for-in enumerate over any object of any class.
You can enumerate over objects, records and interface pointers. The enumerators also can either be objects, records or interface pointers. And the collection members of the enumeration can by any type including fundamentals, compounds, objects, interface pointers, anonymous functions, even class references and type references.
When the compiler sees a code like
for Member in Collection do
.. and Collection is an object, it compiles by the following algorithm..
- Look for a public function of name GetEnumerator(). The function return type must be a class, record or interface pointer type. Protected and below methods do not count. Class functions are also discounted.
- If class, Examine the declaration of the class. There must be public non-class function MoveNext(), with signature…
function MoveNext: boolean;
- Also in this class, there must be a public property named Current. The read accessor method for Current may be any thing, even strict private.
- The type for the current property must be equal to the declared type of the enumerated value (‘Member” in our above example), or, if the type is an object type, the current class type may be a descendant of declared member class. If the member type is an interface type, then the current property interface type may be an interface descendant of the declared member interface type. It is not required for interface types to have syntactically bound guids.
- The MoveNext() and Current are utilised as follows…
With declarations …
TMyEnumerator = object strict private function GetCurrent: TMemberClass; public property Current: TMemberClass read GetCurrent; function MoveNext: Boolean; end; TMyCollection = object public function GetEnumerator: TMyEnumerator; end;
…this procedure …
procedure EnumDemo6a( Collection: TMyCollection); var Member: TMemberClass; begin for Member in Collection do SomeAction( Member) end
… is compiled as if we wrote instead…
procedure EnumDemo6b( Collection: TMyCollection); var Member: TMemberClass; Enumerator: TMyEnumerator; begin Enumerator := Collection.GetEnumerator; try while Enumerator.Next do begin Member := Enumerator.Current; SomeAction( Member) end finally Enumerator.Free end end
If the Enumerator is an interface pointer, then instead of being freed, it is released (reference count decremented). If the Enumerator is a record, then instead of being freed, it is record finalized and de-scoped.
If you are making custom enumerators, you may find it convenient to leverage these base classes and interface pointer types from the VCL…
IEnumerator = interface(IInterface) function GetCurrent: TObject; function MoveNext: Boolean; procedure Reset; property Current: TObject read GetCurrent; end; IEnumerable = interface(IInterface) function GetEnumerator: IEnumerator; end; IEnumerator = interface(IEnumerator) function GetCurrent: T; property Current: T read GetCurrent; end; IEnumerable = interface(IEnumerable) function GetEnumerator: IEnumerator; end; TEnumerator = class abstract protected function DoGetCurrent: T; virtual; abstract; function DoMoveNext: Boolean; virtual; abstract; public property Current: T read DoGetCurrent; function MoveNext: Boolean; end; TEnumerable = class abstract protected function DoGetEnumerator: TEnumerator; virtual; abstract; public function GetEnumerator: TEnumerator; end;
Custom Enumeration over records and interface pointers.
You can enumerate over records and interface pointers too. It works the same way as classes, except that the restriction that the GetEnumerator method be public is not applicable.
Custom Enumeration with helpers
There is a natural synergy between class helpers and custom enumeration. See the next section for examples.
Some incredibly useful examples
Parse a character stream, line by line
With a class helper and custom enumeration, we can conveniently pass a text stream, line by line (or character by character just as easily), like so…
procedure ParseLines( TextStream: TStream); var Line: string; begin for Line in TestStream do ParseLine( Line) end;
One possible implementation to achieve this (with a UTF-8 encoded stream) might be….
TTextStreamHelper = class helper for TStream public function GetEnumerator: TEnumerator; end; function TTextStreamHelper.GetEnumerator: TEnumerator; begin result := TLineReader.Create( self); end; TLineReader = class( TEnumerator) private coTextStream: TStream; ciPosition: Int64; csCurrent: string; protected function DoGetCurrent: string; override; function DoMoveNext: Boolean; override; public constructor Create( poTextStream: TStream); end; constructor TLineReader.Create( poTextStream: TStream); begin coTextStream := poTextStream; ciPosition := coTextStream.Position; csCurrent := '' end; function TLineReader.DoGetCurrent: string; begin result := csCurrent end; function TLineReader.DoMoveNext: Boolean; var Ch, Ch2: Ansichar; sWorking: UTF8String; L: integer; begin sWorking := ''; result := False; while coTextStream.Read( Ch, 1) = 1 do begin result := True; if (Ch <> #13) and (Ch <> #10) then begin L := Length( sWorking) + 1; SetLength( sWorking, L); sWorking[L] := Ch; // Adds a code-unit NOT a character! end else begin if (coTextStream.Read( Ch2, 1) = 1) and (((Ch = #13) and (Ch2 <> #10)) or ((Ch = #10) and (Ch2 <> #13))) then coTextStream.Seek( -1, soCurrent); break end; end; csCurrent := UTF8ToString( sWorking) end;
See my previous blog entry Dances with XML
I haven’t yet developed a class helper for traversing a TDataSet, but it strikes me as something that would be incredibly convenient. I invite the reader to have a go. Tell us about your efforts in the comment stream.
Third party support
Mike Lischke’s TVirtualTree, (a tree and list view component), is probably the worlds’s most popular third party component, and for good reason. TVirtualTree supports for-in enumeration.
I suspect that most readers of this blog are also readers of Nick Hodge’s “A man’s got to know his limitations“ blog. In this entry, Nick introduces the Delphi Spring Framework’s extention to IEnumerable. The entry is worth reading, including Nick’s answer to the originating question on StackOverflow.
- One would expect that the member variable would be syntactically required to be a local variable. Surprisingly this is not the case. However, I advise against non-local member variables, as this is contrary to the semantics.
- One would expect that member variables should not be capturable by closures. And this too is not the case. Again, don’t do it. The outcome would be just too messy.
- Within the loop, the member variable cannot be assigned to.
- The member variable can be referenced, both before and after the loop. Unlike a for-to loop variable, it’s value is indeed defined after loop exit, provided that the loop has passed at least one iteration.