Women listen and men look? How to best communicate risk to support decision making

When my family plans a trip to the beach in Scotland, there is always a high chance of rain. So I always pack everything: from sunscreen to waterproofs. But would I have reached the same conclusion listening to the weather forecast on the radio or when checking the weather app on my phone? And would my husband have done the same?

In a recent study we look into how different types of information presentation will influence the way people make decisions. We are especially interested in the case where there is risk associated, i.e. we make a decision now (bringing waterproofs) based on a prediction of a future event (likelihood of rain). Our study reveals that there are significant gender differences in how people react to different ways this information is presented to them: Women reach better/more accurate decisions when information is presented multimodally, i.e. using natural language text and graphics, whereas men are happy with just the graphics.

Translating numbers into text

But why is there a need to translate numbers into graphics and textual descriptions in the first place? Why can’t we just show the numbers? This relates to the problem of numeracy, which describes how well we can understand numerical expressions. Psychological studies on decision making have found that a high percentage of people do not understand and can’t act upon numerical uncertainty. For example, a recent study finds that about 30% of Germans and Americans are unable to answer the question: “Which of the following numbers represents the biggest risk of getting a disease: 1 in 100, 1 in 1000, 1 in 10?”.

As such, we set out to design a system which can automatically translate numbers into natural language, hoping that verbal/textual paraphrases will make it easier for people to understand the likelihood of potential risks. We have used a technique called Natural Language Generation, which enables computers to produce well-formed, grammatical sentences from abstract representations (numbers in our case). The system imitates human weather forecasters. For example, a 30% probability of rain with sunny intervals will be reported as “Mainly dry with sunny spells”. Further details on this method can be found here.

The MetOffice Weather Game

Fig 1: The MetOffice's WeatherGame

In order to test our system we used extended version of the MetOffice’s WeatherGame
WeatherGame where we measure the impacts of language versus graphics to improve decision making. In the WeatherGame (see Figure 1) the player has to choose where to send an ice-cream vendor in order to maximise sales, based on weather forecasts for two locations over the next 4 weeks. These forecasts describe predicted rainfall (Figure 2) and temperature levels together with their likelihoods using graphics only, text only or a combination. Users then choose one of the locations to send the ice cream van and indicate their confidence. Based on their decisions and their confidence levels, the participants are then presented with their “monetary gain” of selling ice cream.

We got 442 unique players to take part in the game and collected 450 unique game instances (just a few people played the game twice).

Fig 2: Predicted Temperature

Our findings

We find that there are significant gender differences between multi-modal, text, and graphical versions of the task, where for women, use of text results in a 87% increase in task success over graphics. Multimodal presentations lead to a 44% increase for all adults, compared to graphics. People are also more confident of their judgements when using the multimodal representations.

These are significant findings, as previous work has not distinguished between genders when comparing different representations of data. It also confirms research on gender effects in multi-modal systems, as for example reported here. In addition, these results are related to educational research, which suggests that women perform better in verbal-logical tasks than visual-spatial tasks. An interesting investigation for future research is the interplay between uncertainty, risk-taking behaviour and gender, as for example reported here.

The author: Verena Rieser is a Associate Professor/Reader in Computer Science at Heriot-Watt University. Her research investigates data-driven Machine Learning and Natural Language Processing techniques to develop Language Technology.
The research presented here received funding from the EPSRC project Generation for Uncertain Information (EP/L026775/1)and Domain-Independent Language Generation (EP/M005429/1).