Measuring success

TTIX2015 has begun: Day One- Approaches to Research Quality

By 19 February 2015

So, the Exchange got off to an energised start yesterday. More than 200 participants filled the Ballroom promptly at nine o’clock, and were kept engaged, involved, and informed throughout the first full day of the conference. At seven o’clock in the evening those same participants could be found in dynamic discussions with one another over cocktails and ice cream, digesting the issues and inspirations of the day’s sessions.

The Exchange, in its truest sense, has begun.

The facilitator Valerie Traore set a brisk pace for discussions and Ruth Levine, from the William and Flora Hewlett Foundation, set expectations high for the conference to make progress on ‘how to advance research quality while being mindful of the extreme need to be timely, relevant and accessible to non-researchers”.

The over-arching theme “Approaches to research quality” drew a range of views from panels of both researchers and policymakers. There was a plea early on for indicators of quality to be sensitive to context, and for process indicators to consider both quality and utility of research.

Key topics discussed were those which looked at the interplay between internal and external influences on quality. Participants raised questions over internal review structures and how to incentivise researchers to maintain and uphold research quality within an organisation. They were divided over whether publications and payments are effective incentives, but united over the importance of external and environmental frameworks for governing research. “What makes good research?“ And “Who makes this decision?” were recurring themes throughout the day.

An invited panel of policymakers drawn from five countries were asked to describe both how they had been influenced by Think Tanks and what tactics ‘work and do not work’ in engaging and informing them. As befits the complex and non-linear nature of evidence-informed decisions, there were few concrete examples where research triggered visible and direct policy outcomes – but lots of enlightened intermediary steps. These included a more accurate framing of the problem in India, and the trigger for a public enquiry in Tanzania. Examples of what doesn’t work for the panel included single-issue Policy Briefs; research without data; ‘cut and paste’ research that is intellectually lazy; and long emails. “I can read a long book”, said Leonardo Garnier, ex Minister of Education in Costa Rica, “but not a two-page email.” We have been warned.

Balancing relevance and credibility continued to emerge as key themes. And as the day progressed, this balance was unpacked and made explicit. In some aspects relevance provided constraints. As Leonardo Garnier said, in the second session of the morning, “In politics the scarcest resource is time.” Very often, research has to be compact and delivered quickly to enable uptake. This is also true when informing the media and public opinion, where a strong narrative is particularly important. But where speed and brevity is a priority, how can Think Tanks ensure quality?

Participants looked at diversity, building multi-disciplinary research communities, evidence gap-maps and open data. Key aspects continued to emerge: politicisation, incentives, impartiality and relevance were clearly identified as integral to any quality assurance process. Aspects but not answers.

Valerie Traore kept perfect time and the participants left the hall at 5.30pm on the dot to prepare for an evening of reflection and re-energising. For while the answers may not have emerged in their entirety yet, there is something both positive and motivational for the Think Tanks to take from this first day.

They are asking the same questions.

Topics: , , , , ,