Impactful impact
Evaluating the impact of research is not easy but all the same it’s worth taking seriously. Three experts tell you how it pay to evaluate the impact of your own research.
A good idea. That’s the beginning of all research. A question seeking an answer, a theory that’s worth testing, a thought to see where it leads.
But an idea is just the starting point. This is why it may seem challenging, perhaps even downright impossible, to evaluate the significance an idea might one day have. Not even the particle physicists at CERN could probably ever have guessed how, when they were developing it as a secondary project, the world wide web would change practically everything.
“Of course, no-one could have predicted anything like that,” Heli Lehtivuori, Research Specialist at Tampere University consoles us.
“This is why I urge people to think about the impact of research at three levels, from minimum to maximum. The minimum level could simply be what the ongoing research could reveal and where it could lead,” Lehtivuori says.
“Just from the perspective of your own career – if this is successful, then what next?”
The second level is how your own research could impact other research. And it’s not until the third and final level that the change is large-scale: the long-term impact that the research could have if everything falls into place. And that’s a very big IF.”
Stories available
Besides Lehtivuori, our question about the evaluation of impact was answered by Claudia Dell’Era, Head of Pre-Award Services at Aalto University and Emma Pirilä, Director, Research and project services at the University of Oulu. All three have backgrounds as researchers and nowadays help researchers to make funding applications.
The trio know that writing applications does not always come as second nature to researchers.
Heli Lehtivuori
Research Specialist at Tampere University
“A person might feel they wanted to be a researcher but ended up more as an advertiser,” Pirilä sums up.
“It may seem awkward or even disconcerting to think up a story for your own research idea that would appeal to the organization to whom you’re applying for funding.”
All the same, all three assure us that this is definitely worth doing. Since even though you would rather talk about data, test arrangements or uncertainties, when you apply for funding you also need to explain why your research in particular is important and interesting.
“You can’t sell a research project without a story,” Dell’Era sums up.
Nevertheless, the same story won’t work everywhere.
“You can’t sell a research project without a story.”
Claudia Dell’Era
Head of Pre-Award Services at Aalto University
“Before applying for funding, it’s advisable to check what kind of research the funder has earlier supported,” Lehtivuori points out.
“If you can’t find anything that resembles your own idea, then this is probable not the right funder for this particular research.”
It’s always worth tweaking the application slightly depending on the funder. Different funders have different interests and whereas it might be smart to highlight the scientific aspect for one, it might be better to highlight the societal aspect for another.
“It’s also worth warning about not going into too much detail,” Pirilä points out.
“You also need to be prepared for the fact that your application might not be read by the most suitable person. Even though efforts are made to find expert evaluators for applications, in reality perhaps one of them has a deep understanding of the matter. The others understand at a more general level.”
Besides this, evaluators are usually busy. “It’s therefore worth explaining the matter as clearly as possible and not to assume that the reader understands the importance of the research. It may be the evaluator doesn’t understand if you don’t sell well enough.”
Is self-praise any recommendation?
When marketing your own project you also need to think about how much you dare to promise. Dell’Era thinks that Finns traditionally tend to err on the side of caution, and needlessly so.
“If young researchers still dare to envision, professors can sometimes be over cautious. Almost as if they fear that they will lose credibility if not everything goes as imagined.”
“I try to remind applicants that they’re competing in the same call as applications from elsewhere. If you don’t have the courage to make big promises, presumably others will. If you think the research has the potential to make a breakthrough, you have to be bold and say so.”
Lehtivuori reminds us to keep different perspectives in mind.
“When thinking in the short term, it’s worth keeping your feet on the ground but in the longer term you can also envision.” On the other hand, moderation in all things. Lehtivuori says that if overcautious is the feedback for the draft, the next version might be unnecessarily bold.
“However, you always need to have some feeling for what your particular project could achieve.”
“If you think the research has the potential to make a breakthrough, you have to be bold and say so.”
Don’t leave things until the last minute
“When evaluating project impact, it’s a good idea to bear in mind not just the scientific, but also the societal and economic aspects,” Dell’Era reminds us.
“If, for example, you’re engineering a new biofuel, it would be useful to have some calculation as to the potential significance of and market for the fuel.”
The bigger the project, the more important economic and impact issues are.
A single researcher can be awarded money for a very bold opening with many uncertainties. But the larger the group and the longer the funding is applied for, the better justifications are generally required for the results that could be achieved.
“Generally speaking, funding is awarded for research that is thought to be of some kind of benefit.”
Emma Pirilä
Director, Research and Project Services at University of Oulu
All too often, the trio are faced with applications where all the bangs are put into explaining the research idea and evaluation of the project impacts seems to whimper out having been cobbled together the evening before.
“Nevertheless, evaluation of the project impacts is part of the overall application and some of the points are awarded for them. Even a good research plan can fail on poor evaluation of the impacts,” Lehtivuori points out.
Sometimes you have to recommend having another think before submission even if it does mean deferring the application.
“A shoddy application is just a waste of everyone’s time,” Pirilä points out.
“Even so, the ultimate decision rests with the researcher themself.”
Wanted: social significance through works of science and art
As an independent foundation, Jane and Aatos Erkko Foundation wants to enable projects that reflect its values. But what does this mean in practice?
“We’re seeking projects that are nationally or internationally important in their field. They’re ambitious and even risky. Robust, long-term basic research is what drives science forward but projects can also have other beneficial effects,” says the Foundation’s Secretary General Hanna-Mari Peltomäki.
When evaluating applications, we don’t just consider the purely scientific and artistic aims of a project but also its overall impacts, e.g. on the applicant’s own career development or whether overall it strengthens critical skills in Finland.
Updated applicant’s toolbox
During the summer and early autumn, we updated our online service and application guidelines and also published new webpages.
“The updates aim to provide clearer guidelines than earlier for applicants and tools to help them make a good application – we want to help applicants to be successful,” Peltomäki sums up. ”Describe your project clearly and realistically. Sum up your goals and the importance of your project in an understandable way. Explain what makes it special. Evaluation will also take into account your career stage. We also collect more background information to increase our understanding of the sector.”
Aalto Summer School went further than the wood(s)
Jane ja Aatos Erkon säätiön uutiskirje
Uutiskirjeemme tilaajana saat ajankohtaiset kuulumiset sähköpostiisi neljästi vuodessa.
Tilaamalla uutiskirjeen hyväksyt tietosuojaselosteen.