Back when we first were designing the DataGrid we thought we would leave filtering and sorting up to the application. The problem with that approach is that it requires an extra copy of the data. For example, if we want the data sorted by a particular column or set of columns, we need to sort the data outside in the application and reset the DataGrid properties. This creates a whole new copy of the data. Same goes for basic filtering: if we want to select certain rows to display, we need to select the rows in the application and reset the DataGrid properties. This is not good.

It turns out the way we have implemented the grid makes it fairly easy to build in sorting and filtering. Assume the data is a matrix, m. Then at any given moment, the rows that are to be displayed in the available window space are given by a vector of (up till now) consecutive integers i:

      m[i;]

If the user is on the bottom visible row and scrolls down one, then i will be effectively set to i+1. But there is no reason that i needs to be consecutive integers. We can display certain rows in a certain order by simply messing around with i.

In the DataGrid we now track an internal property RowIndices. This defaults to ⍳n where n is the number of data rows. That is, the default is to show all rows in the order given. We can pick out a subset of indices and permute them, assign this to RowIndices and thus display a subset of data in a different order:

      m[RowIndices[i];]

And that's all there is to built-in sorting and filtering. The entire dataset is never sorted, nor is it ever selected out or copied. We limit built-in filtering to selecting where a column is equal to a particular value, or set of values. We can of course filter on multiple columns. More sophisticated filtering, for example selecting rows where a column is greater than a certain value, is left up to the application. There is a new RowMask property, a Boolean of the same length as the data. This allows the application to provide the entire dataset once to the DataGrid once, and then reset the visible rows. Built-in filtering can work on top of application filtering.

We only allow sorting and filtering if the property InsertRows is set to 0, that is the total number of rows cannot be changed. We may relax this in the future.