This is a guest article by Alexander Hogan and Richard Woodall from Etic Lab. Etic Lab is a research and design company that specialises in innovative ways of collecting and analysing data. You can find more information about the company and their work on their website.
Over the past three years, the ARFOR Programme has supported a range of interventions to bolster the use of the Welsh language and to nurture a sense of place and belonging across participating regions. Much has already been achieved, from providing grants to help young people start new businesses to providing funding for projects which seek to strengthen Welsh language use within local communities.
What are we measuring?
But what can we say about the economic impacts of ARFOR? How has increased support for the Welsh language and Welsh culture affected businesses, workers and job-seekers in the participating regions?
A team of researchers including participants from Wavehill Social Research, Aberystwyth University, Sgema and our Powys-based research and design consultancy Etic Lab are currently trying to answer some of these questions with a wide-ranging programme of socioeconomic analysis. Etic Lab’s contribution to this analysis is based on the development and application of new metrics using open web data to measure the use of the Welsh language in the digital presences of businesses and other organisations across the ARFOR region.
The principle behind our work is simple – by monitoring an organisation’s digital presence over time, you can gain profound insights into its priorities, values and internal structure. In the past, we’ve used this method to measure the digital maturity and innovative capacity of Welsh businesses, as well as to analyse how companies are responding to the principles laid out in Wales’s Wellbeing of Future Generations Act. For ARFOR, we’ve built a range of new metrics to evaluate how businesses, charities and public sector bodies are using the Welsh language, as well as to determine the importance of place to how they talk about themselves and their activities.
Below, we’ll give one example of the work we’ve been doing, and how we hope to contribute to the ARFOR Programme going forward.
Measuring the feeling of belonging
A crucial element of the ARFOR brief has been to foster a deeper sense of place and belonging amongst Welsh communities and the businesses that serve them. The aim here is to deliver both social and economic benefits; by stressing the uniqueness of Wales as a landscape and an ecosystem, and Welshness as a vibrant cultural heritage and a shared set of values, it is hoped that Welsh companies will be able to strengthen their offer both to local communities and broader markets.
To address this theme, Etic Lab has developed a novel metric to measure the extent to which businesses and other organisations seek to evoke a specific sense of place through their online communications. This is based on a method known as the Semantic Brand Score (SBS), which was developed by Italian academics in 2018. SBS is a statistical method for calculating the importance of a particular word or phrase within a body of text, based on three factors.
- Prevalence: how often does the word or phrase appear within the text?
- Diversity: how many different concepts does the word or phrase connect to?
- Connectivity: how many different concepts does the word or phrase link between?
SBS doesn’t just capture which words are most commonly repeated; they reveal the role that certain key concepts play in illuminating, contextualising or linking other ideas. For example, a text containing a high prevalence of the term “Welsh values” in connection with ideas like “fairness” and “equality”, and linking together concepts such as “prosperity” and “sustainability” tells you not only about the degree of importance the author places on “Welsh values”, but also something about what that term means to them.
For the ARFOR programme, we have adapted this methodology to help understand how important concepts of place are to businesses and public sector bodies across Anglesey, Carmarthenshire, Ceredigion and Gwynedd. How do organisations in these regions talk about place, and what other ideas do they link it with? Does the “sense of place” mean something different for businesses in Anglesey than it does for those in Gwynedd or Ceredigion?
To build this measure, we used the Enwau Lleoedd Safonol Cymru, or List of Standardised Welsh Place Names, maintained by the Welsh Language Commissioner. This contains not only the names of cities, towns and villages in Welsh and English, but also important monuments and landmarks, allowing us to compile a comprehensive register of the most important place names across the ARFOR region.
Next, we took a structured sample of businesses across each of the ARFOR counties, weighted to reflect the overall economic make-up of each unitary authority (i.e., not just how many businesses each county contains, but what kinds of business, how big and so on). We also compiled a list of public sector organisations for each area.
Each week, we perform a digital scale of the public-facing text on the website of every organisation in our sample to detect mentions of terms from the List of Welsh Place Names. We then determine an SBS score for each one. This allows us to see how often certain place names are mentioned and what other ideas or values they might be linked with, both for individual organisations and at an aggregate level across each region.
In this map, we present some preliminary findings – an aggregate Sense of Place Index score for each of the ARFOR regions, showing the degree to which organisations from each county foreground a specific sense of place in their digital presence. The higher scores for Anglesey and Gwynedd reflect the fact that “being from” Anglesey/Gwynedd is a more important factor for businesses and other organisations in these counties than it is for Carmarthenshire or Ceredigion, perhaps reflecting the former regions’ traditional status as heartlands of Welsh language use and Welsh cultural identity.
As our work continues, we’ll be diving deeper into the data to understand which places within each region are most important to local organisations, and what other ideas or values are associated with particular places.
Etic Lab is proud to be able to contribute to the ARFOR Programme, and more broadly to the project of building a more prosperous, sustainable and inclusive Wales for all its citizens.
