ChatGPT – The next Scots Makar? (...spoiler alert: probably not)

Generative Artificial Intelligence (GenAI) has been gathering steam in recent years, giving rise to more and more publicly available tools. The most prominent of these is ChatGPT: a chatbot which produces ‘human-like’ responses when prompted.

For English – a ‘high resource language’ – ChatGPT does well, producing English which, although the content may sometimes be questionable, is closely aligned to standard written usage. However, recent studies have suggested that for ‘low resource languages’ ChatGPT performance is diminished (Robinson et al. 2023). GenAI relies on a wealth of online data in a given language to ‘train’ itself on: something which many minority languages lack.

Modern Scots is arguably a ‘low resource language’ in terms of online presence. The effectiveness of GenAI content in Scots is also complicated by the lack of an orthographic standard (Bann & Corbett, 2015). In this paper, I will present the results of a small-scale study which uses ChatGPT 4.0 to generate a corpus of ‘artificial’ poetry in North-East Scots (Doric), using a similarly sized corpus of ‘real life’ Doric poetry as comparison points and topic prompts.
Firstly, results regarding the frequency of different categories of words will be presented, comparing differences between the ‘real’ Doric poetry and the ChatGPT texts. These results suggest a more Anglicised form of Scots being generated by ChatGPT and an increased presence of more noncontemporary orthographic features.

Secondly, the GenAI content will be approached with a more critical eye, delving deeper into the orthographic choices. This will focus on the obvious limitations of the ChatGPT function, including noticeable ‘errors’ or ‘non-Doric’ features which appear in the ChatGPT poetry, as well as the degree of internal consistency in the Scots being used.

Finally, possible reasons for these limitations will be explored and suggestions made regarding the future of Scots in this ever-changing digital landscape.