Human-computer interaction: Can you see what it is yet?

check out the video of this interface on

The recent furore over the 2012 Olympics Logo reminds me of how people react to the user interfaces they find on everything they interact with, from websites to washing machines. If an interface, like a logo, is well-designed, no one notices or mentions it. If it is difficult or unsightly, people complain loudly and when given a choice, won’t use an interface they don’t like. Interaction designers, like IT support staff, are never thanked when all is well and severely criticised when interfaces cause users problems. Continue reading “Human-computer interaction: Can you see what it is yet?”

Visualisation: Information is power – just avoid drowning in data

 Map of Great Fire of London Copyright © The British Library Board.
(The Great Fire of London map at the British Library website)

In the 1530s when Henry VIII realised that dissolving the monastries would get him much needed assets, he commissioned a map of London, paying particular attention to ‘lawless’ Southwark. He wanted to see if the borough had any money he could take off them. Henry VIII was a smart man, he knew that the right sorts of information bring wealth and power.

I saw the resulting map last year at the British Library exhibition London: A Life in Maps along with many others – maps of wills and estates, Victorian cab fare maps, cycling maps and tourist maps. Each map was primarily motivated by the need to learn more about an area of London in order to make or save money, especially when making your way around the ever growing London.

In part, this was because travelling around London has always been a daunting thing to do and if you don’t know your way around, you can waste a lot of time and money on convoluted travel. Visualising how everything was connected was impossible to do until Phyllis Pearsall personally pounded the streets, all 3 000 miles of them and put together the world’s first A to Z. She realised that London had to be presented as a cohesive system of streets, buses, and tubes. Like all fantastic ideas, it is so obvious – but back then no one had ever thought of it.

This wasn’t the first time cross referencing different types of data could help people. In 1830’s Dr John Snow used a map of London on which to plot water pumps next to cases of cholera in order to identify how the disease was being spread. He concluded that it was an infected pump near where the outbreak had occurred, and not an airbourne miasma which was the hysterical masses feared. Once he had identified and closed the infected pump, it was easier for Joseph Bazalgette to get the support he needed to build sewage systems which are still used to this day.

Breaking news infographic style

Many online journalists have adopted the same visualisation techniques to present breaking news. They have maps which plot events and eye witness accounts either by video or audio. This information is put up online, as it happens. People get an immediate insight to events good and bad and can make informed make decisions that may save lives.

Techniques range from sophisticated virtual reality programs to simple line graphs. Sometimes this is done well as in the Dr Snow example, other times techniques can be badly employed. Edward Tufte has written extensively about the way to avoid chart junk so that more meaningful graphics can be produced either by hand or as in the case of scientific visualisation by harnessing the power of computers.

In meteorology, molecular modelling, and medicine, computers show us things we could never have seen otherwise: the inside of a bone, or a hairline fracture not detected on an x-ray but visual in a 3D rendered pelvis.

Even football is given a helping hand. Redbee’s Piero software uses live footage from football matches and ties it to a virtual stadium to calculate the exact coordinates of players and the ball during those crucial moments not seen by the referee or caught on camera. Piero creates new virtual camera angles (or viewpoints) and approximates what happened so that the right decisions can be made.

Recognising and reading the patterns

Previously we thought that the field of artificial intelligence would produce systems which could crunch through the numbers alone and present the solution to us. Now we know this is still for the future. We humans are adept at recognising patterns and making links in an almost intuitive manner which is impossible to replicate in a computer. Cognitive scientists try to crack the secrets of the brain and our minds to understand how we reason. Until the time when we can let our machines reason with us, we need to stick with and have control over what we want see.

Football technology is an ideal application for visualisation as there are a limited number of rules to represent and a field of play to be modelled which gives us all of our constraints and the context within which to search. Other applications such as monitoring bridges or the people/traffic flow through a town can be without boundaries so we are left just to literally stare at the waves of data as they happen.

To counter this feeling of being deluged, we need good systems which we as the viewers can interact with and which we, the users, can fix constraints and context. In this space we can then explore and search, using interactions which are translated into algorithms. We exploit and manipulate new viewpoints (like Piero does), we fixate on a viewpoint, we apply analogies and metaphors to find different ways of interpreting what we see and then we transform or combine our data, again to find different viewpoints. The problem is, we sophisticated humans can do this automatically, computers can’t yet, they can only flag up certain patterns that we have told them about.

Maps: Ideal information systems

Exploration is difficult to support, often because the visualisation aspects of software often dominates to such an extent that basic functionality is compromised. Information is presented to the user in a potentially misleading manner and the wrong conclusions are drawn. To counter this, there needs to be a direct correspondence between the human perception of the physical world and the abstract computer-internal representation. Maps are a perfect example of this correspondance and have worked well for centuries. Henry VIII obviously knew how to manage his data.

Google Maps allows you to look at the abstraction of the map and superimpose it on the photographic representation. You can zoom in and look at the trees and houses and still be aware of the street names. This is an accessible form of augmented reality which would have been difficult for the man in the street to imagine on such a widescale i.e., the whole world, even ten years ago. Imagine what Henry VIII would have done with such information at his fingertips, whilst sat in his palaces. He would have zoomed in on Southwark borough and had a magpie’s view of the rich pickings.

Engineer-Computer Interaction for Structural Monitoring

An increased availability of information technology (IT) is currently influencing decisions to monitor structures more frequently. IT is invariably used to interpret structural behaviour from monitoring data. However, engineers remain frustrated with IT results. Engineers work with incomplete knowledge, problem specific characteristics, and context dependency. Although such conditions require interactivity and multiple solutions, most structural monitoring software provides automatic data-analysis with one interpretation.

The aim of this thesis is to determine appropriate computer support for engineers to consider multiple-interpretations so that they can converge upon better explanations of current structural behaviour. This is done through the presentation of an interactive space of solutions to which engineers can add contextual information and delete irrelevant knowledge as necessary. In order to do this successfully, compatible human-computer interaction (HCI) for engineers is needed with suitable visualisations of data and behavioural interpretations.

The main contribution of this thesis is Engineer-Computer Interaction (ECI) which is a new sub-domain of HCI specifically tailored to engineers. ECI provides a generic blueprint which i) uses an information classification schema, ii) integrates standard engineer characteristics and working procedures into the software, and iii) provides a modular approach to task decomposition; for the development of software.

A software toolkit for structural monitoring – Structural Monitoring Tool Kit (SMTK) – has been developed according to the ECI blueprint in order to illustrate ECI utility. SMTK was empirically evaluated through standard HCI techniques to illustrate its compatibility with engineers when performing structural monitoring data interpretation.

Results show that engineers prefer software which has been developed following ECI than existing software because ECI software is tailored to their needs. A second prototype is suggested for in-service prediction to illustrate ECI generality. Finally, areas for future work have been highlighted. In conclusion, ECI is useful for the design and development of software for structural monitoring and has the potential to be applied to other engineering tasks. The ECI extensible foundation leads to revealing comparison between structural monitoring and other tasks such as analysis.


pdf of full PhD

Engineer-Computer Interaction for Structural Monitoring

L’accroissement important du nombre d’outils informatiques disponibles influence actuellement les décisions concernant les auscultations des ouvrages. Invariablement, l’informatique est utilisée pour interpréter les données qui proviennent de la surveillance d’une structure, dans le but d’èxpliquer son comportement. Cependant, les ingénieurs en génie civil restent souvent frustrés par les résultats des outils informatiques traditionnels. En effet, les ingénieurs disposent en général d’une connaissance incomplète du problème, de caractéristiques qui sont spécifiques au problème et dépendent du contexte de la situation. Ces conditions rendent indispensable l’utilisation de traitements interactifs proposant des solutions multiples. Néanmoins, la plupart des logiciels pour l’interprétation de données de surveillance fournissent un analyse des données automatique aboutissant à une seule interprétation. L’objectif de cette thèse est d’apporter une aide informatique appropriée aux ingénieurs pendant la phase d’analyse d’interprétations multiples permettant de converger vers de meilleures explications du comportement effectif des structures. L’approche proposée ici est la présentation d’un espace interactif des solutions que les ingénieurs peuvent modifier par l’ajout d’informations contextuelles o`u la suppression de connaissances inadéquates. Cette approche nécessite une interaction homme-machine qui soit compatible avec les spécificités des ingénieurs, et qui offre des fonctionnalités de visualisation et des interprétations multiples. La contribution majeure de cette thèse est l’introduction de la notion d’interaction ingénieurmachine – Engineer Computer Interaction (ECI) – qui est un nouveau sous-domaine de l’interaction homme-machine con¸cu spécifiquement pour des ingénieurs. ECI fournit un modèle générique qui i) utilise une classification des différentes informations et des différents tâches, ii) intègre des caractéristiques standard des ingénieurs et de leurs procédures de travail, iii) fournit une approche modulaire pour la décomposition des tâches et pour le développement du logiciel. Pour illustrer l’utilité de la méthodologie ECI, une boˆ?te à outils – Structural Monitoring ToolKit (SMTK) – a été devéloppée pour l’interprétation des données provenant de la surveillance des structures. Cette boˆ?te à outils a été évaluée empiriquement en utilisant des techniques standard de l’interaction homme-machine afin de valider la compatibilité de ECI aux ingénieurs. Les résultats montrent que la plupart des ingénieurs préfèrent l’approche de ECI aux possibilités d’interaction offertes par les logiciels existants parce que ECI est spécifiquement con¸cue pour être adaptée à leurs besoins et à leurs exigences. Un deuxième prototype est présenté pour la prévoyance in-situ, ce qui met en évidence la nature générique de ECI. Finalement, des perspectives d’application de ECI à d’autres domaines sont envisagées. En conclusion, la notion spécifique de ECI est utile à la conception et au devéloppement de logiciels pour l’interprétation des données provenant de l’ascultation des ouvrages. La méthodologie ECI peut également être appliquée aux autres tâches du génie civil. Egalement, ECI permet une comparaison pertinente entre la maintenance des structures et d’autres tâches telles que l’analyse des structures.


Full PhD, 2000

Semiotics: It’s a sign!

copyright of

A structural engineer once told me that he would always win pictionary if he was teamed with another engineer. Structural engineers have a symbolic language of their own and use it, normally in the workplace, to communicate more accurately. To the onlooker it is all triangles, little circles and arrows. But to the trained eye they represent bridge spans with fixed supports under uniform loads. Similarly, electrical engineers use seemingly incomprehensible symbols to describe apparatus layout. Continue reading “Semiotics: It’s a sign!”