Redefining clinical meaningfulness

Published on 28th March 2023

A World Dementia Council Summit will this week (Monday) discuss how teams conducting clinical research into Alzheimer’s disease and wider dementia can measure what matters when some interventions clearly benefit patients but may not deliver the dramatic improvements needed to meet current definitions of effectiveness.

The Chief Scientific Officer for Scottish Brain Sciences, Dr John Harrison says the complexity of Alzheimer’s disease and the myriad different ways it presents in different people is often not taken into account by standard approaches to clinical research.

Dr John Harrison

Dr John Harrison

Dr Harrison a psychologist, long term consultant to clinical trials and now CSO of Scottish Brain Sciences is part of a panel discussing ‘clinical meaningfulness’ – the central question researchers ask of any finding in a study: is the effect valuable and important to the patient?

The challenge he says is that nothing is clear-cut in Alzheimer’s disease, and something that’s seemingly normal in one person can be deeply worrying to another:

“The patients we see include individuals in their mid-40s who are still working, who are very sensitive to changes in their cognition. We’ve met actuaries, barristers and a whole bunch of people who, when you measure that cognitively, you can’t find anything wrong, but they say, ‘I definitely know that I’m not as good as I used to be’.

“And I think most people that we would interact with, if they had a very small reduction in their executive skills, they would notice it very quickly. But if you’re retired, in your mid-70s and not working – you probably wouldn’t notice. So, by definition, what’s clinically relevant for Person A is completely irrelevant for Person B.”

To demonstrate the challenges of establishing clinical meaningfulness in brain illness, Dr Harrison uses the example of effect size, a clinical measure commonly used when discussing the outcomes of research how significant a medicine or treatment plan is for patients. Here, a larger positive effect is represented by a higher number between 0 and 1.

While Dr Harrison says this is a very good statistical measure for thinking about clinical relevance and meaningfulness, the threshold often used may differ from the actual impact on patients:

“Most meta-analyses, when you look at what’s the threshold for de-facto clinical relevance, it’s an effect size of 0.5 – so that seems to be the threshold for saying that we can assume it actually does make a difference for people.

“The challenge comes in that almost all the drugs that we work with for Alzheimer’s disease, these have an effect size between 0.3 and 0.5 so it’s never that they’re completely useless but it’s never that they’re conspicuously worthwhile.”

Dr Harrison says context is also important – for researchers, clinicians and patients:

“I’m always very keen to say it’s contextual – these may sound like ‘small’ effect sizes for these drugs but, for the right person, it’s truly the difference between carrying on working and not. And that’s about as clinically relevant I think anything could possibly be.”

Dr Harrison emphasises that measuring small changes matters; when a person moves from mild cognitive impairment to Alzheimer’s disease, the previous measurement and monitoring separate functions seems to end:

“All that differentiation goes away, and this completely misses the point because if I’ve got somebody who’s presenting with executive dysfunction [task-related problems], that’s a really helpful thing to know about them in terms of their management.”

“In clinical trials, executive dysfunction is only rarely measured. The primary outcome measure used is to look at memory, praxis and language, but there’s no testing of attention, working memory or problem solving.”

This standard test used in clinical trials, called the Alzheimer’s Disease Assessment Scale-Cognitive Subscale (ADAS-Cog), was developed in the 1970-80s and is regarded as the “gold standard” for assessing the efficiency of antidementia treatments.

Yet, Dr Harrison says it’s not up to scratch when considering clinical meaningfulness for people in these early stages:

“It’s so insensitive that people score perfectly on the language and praxis items, even though we know they have a deficiency.

“So, every time a drug shows positive effects of on the ADAS-Cog, I get rung up and I say it’s a memory effect. And they ask, ‘how’d you know that?’ and within a mild population, it couldn’t be anything else because the only tests they’re impaired on are going to be the memory tests.”

Dr Harrison adds that, using this measure, a ‘positive effective drug’ for Alzheimer’s disease is essentially a pro-memory drug. While that remains helpful in the realm of memory, difficulties arise in meeting clinical relevance and clinical meaningfulness for patients impacted in other ways.

He says monitoring the key areas of cognition when people first attend primary care will require an improved assessment – that the current test is based on Alzheimer’s being a memory disorder despite understanding of the disease moving forward since it was first used. This means only part of what matters is measured, and the impact of an intervention on other areas is missed:

“We need a better way of picking up the diversity of that first presentation in primary health care. Then we need to be developing outcome measures in clinical trials that actually index these things – rather than just ignore them.

“So the first order of business is characterising presentation, because there’ll be some lessons there. And then when we’re looking to fix it, make sure we cover the gamut of cognition.”

He adds that – equipped with the knowledge and research that is already available on the diversity of presentations and levels of cognitive impairment – regulators could be more prescriptive in their regulation of drugs.

Dr Harrison highlights that establishing cognitive clinical meaningfulness is a particular problem when it comes to Alzheimer’s treatments:

“If I look at other indications – we don’t have this problem with depression or dementia with Lewy bodies. It’s peculiar to Alzheimer’s disease, where there’s this strange legacy.”

He says deciding what represents clinical meaningfulness of a treatment may ultimately be down to regulators. But he adds that fixing the system means including more people in research, measuring much more over longer periods, and establishing exactly what matters to patients – an approach that is central to the ambitions of Scottish Brain Sciences.

Read more:
Dementia in a new era: summit 2023 | World Dementia Council

Latest news and research

Redefining clinical meaningfulness

Published on 28th March 2023 by Digital Essence

One of the world’s foremost researchers into Alzheimer’s Disease and the wider diagnosis and treatment of dementia says, for the last century, investigations into these conditions have been focusing on the wrong place.

The Founder and Chief Executive of Scottish Brain Sciences, Professor in Psychiatry of Ageing Craig Ritchie, has been telling a seminar in Aberdeen that science and medicine need to move away from the concept that neurodegenerative disorders are conditions of older people and, instead, regard them as treatable conditions of midlife:

Craig Richie

Craig Richie


“It’s a bit like cancer and heart disease, your illness is progressing long before you start getting symptoms. It is the diseases that lead to dementia that we need to be identifying and treating because, by the time people are showing symptoms, their disease is at a very late stage.”

Professor Ritchie was revealing more details about Scottish Brain Sciences at a meeting arranged by Opportunity North East Life Sciences in Aberdeen.

The company has been set up to generate new opportunities for people to get involved in clinical research into brain health and brain diseases, with people soon to be invited to enrol themselves in brain health research.

Professor Ritchie says central to the organisation’s work will be a network of brain health clinics:

“The first thing people need is somewhere they can go with their concerns about their brain health, be it because they have a family history, very early symptoms or simply a feeling that things are not quite right anymore. These answers will be offered through brain health clinics.

“We already know that, for around four in ten people, their disease may be modifiable. So, for people attending a brain health clinic where we can do brain scanning and blood testing, we will create the opportunity to have their own risk factor analysis and to create for them a personalised prevention plan – their own strategy to reduce their risk of developing dementia.

“For those who are diagnosed, pharmacological interventions are going to play a central role, And, because these drugs work best when the disease is caught in its infancy, we hope people will get involved at a much younger age than has previously been the case.”

The Opportunity North East seminar in Aberdeen, which coincided with the Alzheimer’s Research UK Conference in the city, brought together the triple helix of clinicians from NHS Grampian, academics and students from Robert Gordon and Aberdeen universities, and people from a mix of life sciences companies in the region.

Professor Ritchie says he was pleased to use this meeting to talk in detail about the challenges and share how Scottish Brain Sciences wants to work in partnership with NHS, academic and industry colleagues to address them:

“I am an Aberdonian and graduate of Aberdeen medical school, so I am proud of what is happening in the North East.

“If Scotland is leading the world in the brain health movement, which it undoubtedly is, then it’s particularly gratifying that Aberdeen should be furthest ahead in Scotland.”

Read more:

Briefing: Scotland’s ‘new story’ for dementia policy
Scots women at the forefront of medical sciences
Music’s potential to help with dementia

Latest news and research