Evaluation Summary Table


There are two sides to evaluation. One is to evaluate an existing design to determine if it uses best practices for design and implementation. The other is as part of an ongoing spiral design process. Evaluation can be used to determine what should be done next with an evolving design to improve its cognitive efficiency.

To use VTDPs for evaluation consider if each in turn is relevant to the design problem. Next determine if best practices are being followed and implement them if they are not. For example, if the same data is provided in multiple views, then VTDP Brushing should be implemented unless there is a good reason not to do so. Visualization design is complex and so VTDPs do not provide hard and fast rules, but in general they are likely to provide guidance towards perceptually and cognitively efficient visualizations.

The table below shows some of the more basic points to be considered in an evaluation process.

VTDP Design Principle
VTDP Visual Query Graphical design should enable rapid execution of all task relevant visual queries. This means that important symbols should be easily found through visual search, links easily traced visually in network diagrams, and related categories of symbols perceptually grouped.
VTDP Visual Monitoring Applicable to visualizations for ongoing monitoring where an operator has additional tasks. If a strict schedule is needed, computer based reminders should be implemented. Motion coding can be an excellent visual cue for providing operator interrrupts because motion is visible in the periphery of the visual field.
VTDP Cog reconstruction Applicable if a visualization tool supports work sessions separated in time. The visualization should support cognitive reconstruction by providing annotation support as a memory aid and enabling the restoration of system state.
VTDP Drill down All symbols should support drill down if they represent undisplayed task relevant information.
VTDP Path finding Applicable If the task involves tracing network paths. If average visual path tracing takes more than 10 sec interactive support should be considered.
VTDP Drill down with hierarchical aggregation Applicable where moderately large data sets are inherently hierarchical or where there is a natural hierarchical decomposition. To support cognitive efficiency adequate information scent should be provided to support decisions about which aggregated objects merit drill down actions.
VTDP Seed then grow Applicable if the task involves understanding the local neighborhood of a very large network in the vicinity of a particular starting point (e.g. a person). An object is clicked on and related objects appear in a network view. To support cognitive efficiency adequate information scent should be provided to support decisions about which aggregated objects merit growing actions.
VTDP Find local network patterns Use neighborhood highlighting if the task involves finding local network pattern in a network that is of medium complexity 30>n>500 nodes n>l>2n links.
VTDP Pattern comparisons Support side by side comparisons using magnifying linked windows or snapshots if the task requires frequent pattern comparisons in a moderately large information space. The decision about whether to implement extra windows or a snapshot gallery should be based on the capacity of visual working memory the visual complexity of the patterns to be compared, and the frequency with which they must be compared.
VTDP Cross view brushing Applicable if data entities are displayed simultaneously in multiple views, and if the symbol referents are not clear, cross view brushing should be applied. Rapid feedback and visually salient highlighting should be used to support perceptually efficient comparisions between the views.
VTDP Dynamic queries If a set of data entities are too large to display clearly and require frequent filtering for display dynamic queries should be considered if there is a set of scalar attributed that provide a task relevant basis for filtering. To a rough approximation each filterable attribute can reduce the size of the data set on the display by a factor of 10. So if there are filterable attributes the data set may be reduced by a factor of 10,000.





    Visual Thinking Design Patterns are partially funded by the DARPA XDAPA project