How to improve Sorting performance on a None-Virtualized DataGrid?

I am sure most of you would now be surprised why we had to turn off the Virtualization for the wpf datagrid. While its true that virtualization helps to reduce memory footprint, it adds up CPU overhead and the scrolling experience is not flawless.

Due our client's request, we had to disable the virtualization in the datagrid and optimize it further and now it scrolls very smoothly up and down without any lag. The disadvantage is that the data is pre-loaded and kept in memory. This is a solution we can live with.

However the sorting has now become a big problem. While it's true that using a CustomSorter : IComparer would be a better sorting alternative to the usual SortDecriptors, it barely makes any difference in our case though as the whole rows are being redrawn.

Is there any way to improve sorting speed on a non-virtualized datagrid?

Highly appreciated,

UPDATE:

I have come across an idea, which I am trying to implement. Unbinding the Itemssource, do the sorting and once the sorting is finished, rebind the Itemssource.

In order to achieve this I am deriving from DataGrid to capture the SortHandler (that is when the column is clicked by the user)

public class CustomSortDataGrid : DataGrid
    {
        public CustomSortDataGrid()
        {
            Sorting += SortHandler;
        }

        private void SortHandler(object sender, DataGridSortingEventArgs e)
        {
            DataGridColumn column = e.Column;
            IComparer comparer = null;

            // prevent the built-in sort from sorting
            e.Handled = true;

            ListSortDirection direction = (column.SortDirection != ListSortDirection.Ascending) ? ListSortDirection.Ascending : ListSortDirection.Descending;

            //set the sort order on the column
            column.SortDirection = direction;

            //use a ListCollectionView to do the sort.
            var lcv = (ListCollectionView)CollectionViewSource.GetDefaultView(ItemsSource);


            comparer = new BidYieldComparer(direction);

            //apply the sort
            lcv.CustomSort = comparer;

        }
    }

This would utilize the faster Comparer Sorting that is superior to the SortDescriptors. Now teh question is at what stage do I unbind the items sort, apply the sorting, wait for the sorting, once event (which one?) fires then rebind the Itemssource to the view.

BindingOperations.ClearBinding(this, ItemsSourceProperty);

This line above would clear the binding.

//apply the sort
            lcv.CustomSort = comparer;

And theoretically (unsure if this is the right way) ItemsSource = lcv; would rebind it. But the performance is still the same. :(

Any idea anyone?