Back to blog

How can we … understand the role of AI and social networks used by public intellectuals?

Ehab Shanti, PhD graduate/researcher, Department of Sociology

03 October 2023

Categories:

Accelerate spark data science residency

When we first added friends and updated our profiles on MySpace in the early 2000s, few of us could imagine the impact social media would have, including the rise of powerful ‘influencers’ who shape popular views about everything from fashion trends to politics.

I use data science to examine the impact of social networks on human intelligence and specifically, intellectual life. I’m especially interested in how the discourse set by a new type of intellectual sharing ideas online differs from that of academics debating ideas before the Internet.

There is a widely-accepted view that the discourse has changed. Opinions and ideas certainly travel fast online and human beings are undoubtedly producing more data than before, while machine learning and recommendation algorithms have a pervasive influence over much of our lives, but we still know relatively little about the sociological impact of this communications revolution . Indeed, it is an area of study with vast research potential.

A brave new world

My hypothesis is that existing sociological frameworks cannot sufficiently capture the new type of public intellectual, propelled to fame (or at least into select followers’ consciousness on Twitter) by the communications revolution. I’ve examined this phenomenon across three central pillars.

I began by attempting to locate intellectual influencers on Twitter and juxtapose their popularity with their academic attainment. I found that this new type of intellectual is an extreme outlier in the world of social media influence but has an average (or less) position in academia. This conclusion was reached by scraping the Google Scholar profiles of over 50 academics from the same field and comparing it with data gathered on their online presence. When one compares the digital online footprint (e.g., measured by the number of followers on Twitter) of the new breed of intellectual compared with 50 other academics, one sees the presence of an extreme power law, wherein the vast majority have little to no engagement with social media.

Secondly, I applied Natural Language Processing (NLP) tools and qualitative analyses to examine how the discourse may have changed due to social media. I specifically used deep learning to look at the semantic connections between words used in social media posts by intellectuals on Twitter.

This approach revealed that the discourse has become more rapid, engaged, and viral, yet frequently superficial and, in many cases, uncivil (e.g., the proliferation of misogyny and online bullying).

Lastly, I used network science tools such as the NetworkX library by Python to investigate the new intellectuals’ networks and political ecosystems. This involved examining the role that recommendation algorithms and AI itself may play in galvanizing the phenomenon and creating polarised communities.

Finding the unexpected in unknowns

The sheer complexity of the emergence of a new type of intellectual sharing ideas online necessitated an interdisciplinary approach to the research. I gained practical skills to conduct advanced digital analysis methods such as machine learning, deep learning, and NLP thanks to the Data for Science Residency and the Machine Learning Academy courses offered by the Accelerate Programme. These also provided me with a central foundation of knowledge to examine the role that AI and algorithms may play in social networks.

Using deep learning NLP to analyse Tweets revealed some surprising findings. Whereas statistical analysis can reveal some basic connections or give us a median, machine learning and deep learning can reveal patterns undetectable to humans. Deep learning algorithms can find patterns among ‘knowns’ and ‘known unknowns’ within data, and this persistent iteration and clusterisation can reveal ‘unknown unknowns,’ or patterns that human researchers did not anticipate.

For example, NLP and topic modelling revealed that one intellectual vlogger, known for talking about a certain topic on YouTube is more concerned with promoting themselves than discussing their topic of expertise. While data can give us clues and inferences, this technology can provide evidence to support or negate them. I believe it’s an essential piece of the toolkit for sociologists trying to understand digital media.

Deep learning for better findings

I firmly believe that the complexity of social media interactions fundamentally calls for an interdisciplinary approach that includes not only sociological theoretical frameworks but also embracing new empirical methods of data science.

It’s hugely beneficial for sociologists to leave their comfort zone and acquire basic skills and knowledge about data science and machine learning. Equally, data scientists can benefit from the vast corpus of social theories that were developed over centuries of research, including empirical experiments.

Data and network science research, especially concerning human interactions, presents profoundly complex socio-ethical, epistemological, and philosophical questions such as: what data has been created or collected by whom and for what purpose? Using which parameters and to whose benefit? My future work will be dedicated to exploring such questions with an interdisciplinary lens anchored in social theory and empirical sociology whilst not shying away from exploring and demystifying data science and algorithmic logic.

I’m captivated by human networks and believe artificial intelligence will reveal fresh insights into the sociological implications of the communications revolution and decode what’s happening socially on social media.