Analyzing Social Interaction with the help of Computer Vision

A bit of self-promotion: Together with my colleagues Doron Shiffer-Sebba and Yoav Goldstein, I just published an article in Sociological Methods & Research outlining a new way to analyze social interaction. We call the approach “3D Social Research” (3DSR). In a nutshell, 3DSR uses CV and 3D camera footage to study kinesics and proxemics, two …

Continue reading “Analyzing Social Interaction with the help of Computer Vision”

Two surveys on Social Media Use

The PEW research center conducts surveys on social media use in the US that provide some useful information for researchers using online video data in their research. For instance, the surveys can help illustrate the importance of online videos as data in current and future social sciences research.

Online Video Data and Consent

Using online video data for research can entail complex issues of research ethics. Only because a video is accessible on public platforms such as YouTube or TikTok doesn’t mean that the people visible in videos gave consent to being filmed and the video being uploaded. A recent example of this was reported today in this …

Continue reading “Online Video Data and Consent”

Data Set on Mask Wearing Compliance in New York City

The Metropolitan Transportation Authority (MTA, New York City’s public transportation provider), provides observational data on mask wearing compliance. This could be an interesting data source for researchers working on Covid and social interactions in public spaces (see, for instance, this paper and this paper by Marie Lindegaard, Wim Bernasco, and colleagues). Among other possibilities, the …

Continue reading “Data Set on Mask Wearing Compliance in New York City”

Reconstructing a racist terror attack

Video Data Analysis can be used for the detailed analysis of all types of events, including terrorist attacks. The NGO Forensic Architecture published a visual investigation of the racist terror attack in Hanau, Germany, on 19 February 2020, where nine people were murdered by a gunman. They combine and synchronise six CCTV camera perspectives and …

Continue reading “Reconstructing a racist terror attack”

Thermal Imaging as a Tool for Social Research

Video technology keeps evolving, and this constant evolution creates new tools for social scientists to collect data and study situational dynamics during events and processes. One such innovation is thermal imaging cameras. Thermal imaging recordings are able to capture states of body heat and changes therein. This allows researchers new insights into people’s emotions, because …

Continue reading “Thermal Imaging as a Tool for Social Research”

Combining Video Data and Computational Methods

It has been a few months since the attack on the US Capitol on January 6, 2021 that left five people dead and triggered a second impeachment process of then-President Donald Trump. But it is worth revisiting the event as video researchers, because it provides a perfect example of the potential of how we can …

Continue reading “Combining Video Data and Computational Methods”

Video data is “always on”

In his 2018 book “Bit by Bit,” Matthew Salganik describes one feature of digital trace data (such as Twitter or Facebook posts) as being “aways on,” meaning that such data are produced continuously without any researcher having to design a project, acquire funding, and start data collection. One exciting consequence of this is that digital …

Continue reading “Video data is “always on””

Fake videos and how to detect them

A CNN article highlights how to distinguish deep fake videos from realones. They suggest checking for specific natural details, spottingpotential mismatches, and watching out for clues in the behaviour of theperson filmed or background. This can offer interesting insights for VDA researchers who use “ready-made” video data from online sources (i.e., video data they did …

Continue reading “Fake videos and how to detect them”

Policing and crowd dynamics: An NYT visual investigation

The New York Times conducted a visual investigation on police use of force during a protest in Philadelphia in June 2020. The reporters use overhead video footage, numerous recordings from mobile phones on the ground, the video of a police car dash cam, area maps, interviews with protesters, and police documents. The visual investigation nicely …

Continue reading “Policing and crowd dynamics: An NYT visual investigation”

How video-based research can help us understand social dynamics during the COVID-19 pandemic

We just published a short Essay on Oxford University Press’sAcademic Insights for the Thinking World. In it,  we discuss two ways in which social science research can study human interaction during the Covid-19 pandemic. How do people adapt to social distancing and mask-wearing? How do interactions unfold during video calls?

Newsday documentary films interactions with Long Island real estate agents

Newsday published an investagation into racial discrimination in the housing market. They trained 25 undercover testers, who interacted with 93 real estate agents and secretly recorded 240 hours of meetings. The investigation is not only an indictment of unfair practices in the housing market, it shows interesting new ways to gather observational data in real-life …

Continue reading “Newsday documentary films interactions with Long Island real estate agents”

The New Jersey Families Study – A potential data source for research on education, learning, and child development

This may be an interesting study for people working on education, learning, and child development: A team at Princeton collected video and survey data from over 20 families in New Jersey, filming interactions in their homes over a two-week period. As we understand, the data will eventually be accessible for other researchers. If you are …

Continue reading “The New Jersey Families Study – A potential data source for research on education, learning, and child development”

Verification of videos and other online data

In the age of online data, researchers and journalists face some of the same challenges when it comes to ensuring the veracity of a piece of data or source of information. Here’s a toolset that journalists use to gather and double-check online information. Maybe it includes some useful tools for researchers using videos and other …

Continue reading “Verification of videos and other online data”

New journal dedicated to the video-based study of social interaction

We recently learned about a relatively new journal, Social Interaction. Video-based Studies of Human Sociality. We have not yet published with this journal, but it may be a potential destination for publishing some types of VDA research. The journal also looks like a great resource for insights into video-based research. For instance, in a recent …

Continue reading “New journal dedicated to the video-based study of social interaction”

VDA YouTube channel launched

The new VDA YouTube channel features interdisciplinary research talks on novel (qualitative and quantitative) ways to use video data in the social sciences. It includes talks by sociologists, education researchers, computer scientists, psychologists, criminologists and others. We will keep adding new videos and talks in the future, so keep checking in!

Video Surveillance Footage Shows How Rare Violence Really Is (Oxford University Press’s Academic Insights for the Thinking World, OUPblog).

Today videos from closed-circuit television, body cameras, police dash cameras, or mobile phones are increasingly used to capture violent events. Yet, examining footage of violent situations – from the very cameras set up because we believe that violence lurks around every corner – shows how rare violence really is. Continue reading.

Newest developments in the field of AI-based video alteration (The Guardian)

As AI becomes an increasingly powerful and accessible tool to change video content or create entirely fabricated videos from scratch, such deepfake videos will become an increasing issue for social scientists. In this article, The Guardian reports on a new app enabling users to create deepfake videos within minutes.

Visual investigation in journalism (The New York Times)

In a major escalation of violence since protests began four months ago, the Hong Kong police fired a live round at a protester from point-blank range. The New York Times analyzed the footage to create a comprehensive picture of what exactly happened.

How Surveillance Cameras Could Be Weaponized With A.I (The New York Times)

As social researchers interested in video data, computer vision approaches to video analysis are an exciting prospect. However, the potential dangers of powerful automated video analysis are all too real, as this article in the New York Times,  explores in reference to a report by the American Civil Liberties Union.