Touch the screen, receive an odor, and choose its name from four options. In 2024, Marcio Nakanishi and colleagues evaluated MultiScent-20, a 20-item odor-identification test using a dedicated tablet of this kind.
The aim was not simply to build an odor-delivery device, but to investigate what the test measures and how accurately. Using results from 1,832 people, the researchers analyzed item difficulty and the ability to distinguish levels of identification ability, while also examining provisional standards for interpreting scores.
- People answer the questions; the device does not measure smell directly
- Analyzing 1,832 people and building norms from 1,299
- Examining item properties, not just the number correct
- More information at lower ability levels
- An association with education
- Comparisons with patient groups and an existing test
- The value of automation and the challenge of different cultures
- Conclusion
- References
People answer the questions; the device does not measure smell directly
MultiScent-20 combines a seven-inch screen with an odor-delivery system. Originally developed to present fragrances in the cosmetics and fragrance sector, the device accepts cartridges containing 20 odors.
When a participant presses an on-screen button, air passes through the cartridge and an odor emerges from an opening for five seconds. Participants choose the best matching name from four options for odors such as coffee, vanilla, rose, and garlic, then enter their answer on the device. Recording and scoring are automatic.
The test measures the ability to identify odors by linking them to names. It does not measure the detection threshold for increasingly faint odors or fragrance preferences. Nor does a sensor in the tablet directly measure a person’s body.
Analyzing 1,832 people and building norms from 1,299
Participants were native Portuguese speakers aged 18 or older, recruited from hospital staff, students, administrative staff, glass-factory employees, and other groups. This was a convenience sample, largely including factory workers, rather than a random sample of the population.
For the analysis of whether the test could measure a broad range of abilities, people with recent colds or self-reported reduced smell were not uniformly excluded. Including responses from people with differing abilities helps establish which range of differences each item can capture.
By contrast, when constructing the reference score distribution, the researchers excluded people meeting criteria such as recent infection, reported reduced smell, or head injury. This analysis used 1,299 people; the mean score was 17.5 out of 20 and the median was 18.
That average is not a universal normal value for every age or culture. The authors themselves describe the norms as provisional, to be updated as more diverse data become available.
Examining item properties, not just the number correct
The central analysis used item response theory. Even within the same 20-item test, an item that is easy for nearly everyone contributes differently from one whose probability of a correct answer changes substantially with ability.
This approach examines the relationship between identification ability estimated from a participant’s full set of responses and their probability of answering each item correctly. Of the models compared, the best fit allowed two properties to vary by item: difficulty and discrimination between ability levels.
Coffee, for example, was relatively easy, while rose was difficult. The probability of answering the vanilla item correctly changed substantially with ability, whereas rose was less effective at distinguishing ability differences.
This does not mean that coffee smells strong to everyone, or that someone who cannot identify rose has poor smell overall. These are properties of the items under the study’s odor-presentation conditions, response options, and participants’ experience.
More information at lower ability levels
Across the test, measurement was more precise in the below-average range of identification ability. “More information” here means that responses allow relatively fine distinctions between people within that ability range.
People of average ability can answer many items correctly, so scores tend to cluster around 17–18. The authors therefore suggest that the test may be better suited to assessing reduced identification ability than to distinguishing small differences among people with high ability.
Some items, including coconut, tire, and strawberry, had problems fitting the model. The findings also point to room for adjusting odor concentrations or item design.
An association with education
The primary-education group scored about one point lower on average than the secondary- and tertiary-education groups. The association with education remained in an analysis that also considered age group and sex.
Odor identification involves not only detecting a stimulus with the nose but also knowing its name, prior experience, and understanding the response options. This association does not establish that education itself causally changed the sense of smell.
Associations with age group and sex were not statistically clear in this analysis. That does not mean age and sex are unrelated to olfaction generally. The participant population was limited, and these factors explained only a small proportion of variation in scores.
Comparisons with patient groups and an existing test
To investigate validity, the researchers compared 55 healthy people without smell complaints with 13 people with conditions such as smell impairment associated with chronic rhinosinusitis. Mean scores were 17.8 and 7.0, respectively, a large difference. However, the groups also differed in mean age, so not all of the score difference can be attributed to disease alone.
A separate comparison included nine patients with Parkinson’s disease and eight controls matched for age and education. Patients scored lower. Among these 17 people, scores also correlated strongly with the existing 16-item Sniffin’ Sticks odor-identification test, with a correlation coefficient of 0.88.
These findings support the test’s ability to capture differences in identification ability. They do not show that a tablet score alone can diagnose rhinosinusitis or Parkinson’s disease. The correlation with the existing test was established in 17 people, not all 1,832.
In an assessment repeating the test in the same people approximately one month later, initial and repeat scores had a correlation coefficient of 0.94, and another measure of agreement was also high. This supports stability over an interval, but further work is needed to establish how accurately the test tracks long-term change or treatment effects.
The value of automation and the challenge of different cultures
Controlling odor-delivery duration and recording answers automatically can reduce the examiner’s workload in presenting and scoring each item. The authors suggest possible benefits for large studies and repeated measurements in the same people.
Knowing each item’s difficulty and the ability range where it is informative could also support adaptive tests that select the next item based on earlier responses. For example, answering an easy item correctly could lead to a harder one, gathering relevant information efficiently. This study did not demonstrate the effectiveness of a completed adaptive test.
Although the intention was to select widely familiar odors, the participants were predominantly Portuguese speakers. An odor or name familiar in one culture may not be equally recognizable in another. Cross-cultural validation and broader reference populations are needed.
The study was funded by the device manufacturer Noar. Nakanishi disclosed serving as a medical adviser to the company and receiving lecture fees, while Galvão disclosed being its CEO and a shareholder.
Conclusion
The dedicated tablet delivering 20 odors showed preliminary validity as an odor-identification test and stability on repeat testing. Examining individual item properties also showed that the test is particularly informative at lower levels of identification ability.
The important advance is not merely delivering odors from a digital device. It is standardizing presentation and recording, and understanding which items help measure whose abilities. The authors suggest that developing this foundation through validation in diverse populations and adaptive testing could make olfactory assessment more efficient.
References
Nakanishi, M., Brandão, P. R. de P., Freire, G. S. M., et al. (2024). Development and validation of the MultiScent-20 digital odour identification test using item response theory. Scientific Reports, 14, 15059. DOI: 10.1038/s41598-024-65915-3. Full text. © The Author(s) 2024. The original article is available under CC BY 4.0.
