Performance Tuning for PAW UI

As we near the final release stage for our version 1.0 product, the last few releases will be focused on performance. We have been tweaking the UI, so that the user gets into the product and is able to start using it easily. This caused us to move a lot of the functionality to a web-based paradigm - using browser controls for the menuing, displaying data, as well as performing certain actions (filtering, sorting).

Web-Based Data Display
So we decided to focus on the web-based data table as the primary go-forward option. We made it the default for showing large sets of data resulting from running processes. It was already the default for tables that one can edit, but processing results tend to be much larger and hence it is imperative that the user can see and navigate through the data quickly. Hence the focus on performance.

In terms of performance enhancements, we reduced the time to display large datasets from 50+ seconds for 3k+ rows (when using the web-based rich table) to 7-8 seconds. This is inline with the native windows table w/o the rich functionality (multiple lines in a table, filtering, etc..).

This was accomplished using DocumentFragment for DOM manipulation so that we are not causing the browser to re-flow the document when each cell is added. Many thanks to John Resig (http://ejohn.org/blog/dom-documentfragments/) for this helpful hint. It made a almost 10x improvement, beyond the 2-3x improvement that's been noticed for smaller amounts of DOM manipulation.

Interactive UI
Process runs have also been moved to being "jobs" which run outside of the UI and then update the UI once the data has been refreshed. This allows the user to continue to perform other actions. Again, this makes the application more responsive to user input. Similarly, we have added a preview when importing data from the web, allowing the user to see where a page might have changed in case the web import isn't working as expected. This is also provided in a separate window.

As next steps, we will be looking at tuning some of the more resource intensive operations like merging large datasets through the use of more advanced algorithms and more specialized use of RAM.