All posts by Magnus Andre Knustad

Researching comments on news articles

In my master thesis research I am looking at comments on news articles on news websites and on Facebook. During this process I have had to find a way to categorize comments. This has been a long, and sometimes frustrating process. I started out, before beginning the research, with a few categories that I thought would cover all comments, but that I expected would have to be changed. And they were.

I have implemented a heuristic research approach. The qualitative heuristic approach is an exploratory research method where the data affects the categories. This means that the researcher should be “open to new concepts and change his preconceptions if the data are not in agreement with them” (Kleining and Witt 2000). This lead to the creation of 5 broad categories, that I for a long time was quite happy with.

But then I decided to be thorough (stupid me), so I did a reliability test. I gave a person not involved with my master thesis a list of 60 comments to categorize, and then compared them with my own categorization. The results were not good: 60% agreement. After tweaking the definitions of my categories, creating a more procedural method of categorization, and coding the comments again, a second and third reliability test gave me a score of 79 and 75%.

A reliability score of 70% or more is often used as a criterion for exploratory research. But a score of 80-90% would be considered more acceptable in most situations, and above 90% is considered acceptable in all situations (Lombard, Snyder-Duch and Bracken 2002, 593).

Things were looking better, but I was still not happy. So I decided to start completely from the beginning with a new method for finding categories. In stead of coding the comments into one category or another, I wrote any word that came to mind when reading a comment. After doing this with enough comments (about 100) a pattern emerged. Some words were repeated, and these word became the foundation of my new and final categories. And with these categories, another three reliability tests gave me a score of 82-92%. And the categories, which I now am quite confident about, are:

 

Argumentative comments: Arguments contain a proposition that can either be true or false (Blair 2009, 44). These propositions should be testable. They are also formulated for the purposes of persuasion. This means that there needs to be a point of view that the commentator wants someone to adopt, that is backed up by a proposition. Argumentative comments can also contain proofs from classical rhetoric: Ethos, Logos or Pathos.

Opinions: Opinions are comments that are not necessarily meant to persuade, but function as a direct or indirect statement of what the commentator thinks and believes about an issue.

Reactionary comments: Reactionary comments are short expressions of emotions with little or no informative value. They can also be unspecific statements – statements that are not specific enough for the reader to accurately interpret what the commentator is writing about.

Reactionary comments often contain a set of punctuation marks, especially the exclamation mark, or sets of emoticons. Reactionary comments can also be non-verbal. In these cases, the comments contain either only emoticons or written non-verbal expressions, such as “Haha!!”, indicating laughing or joy.

Informative comments: These comments do not directly argue for or against something, although they can be used in discussions to build a case for a point of view. They are meant to provide relevant information – whether or not that information is factual. Informative comments, with the exception of those classified as personal experience, contain testable factual information that can be either true or false.

Derogatory comments: These are comments that uses some form of critique or potentially hurtful discourse, usually directed at another commentator. Put they can also be directed at public figures or the subject of the article.

Humorous comments: These are comments that, with the intention to be funny, brings together two disparate ideas, concepts or situations in a surprising or unexpected manner, or that contains a play of words or self-deprecating, humorous statements.

Tagging comments: Comments containing only a tagged name. They are usually found on Facebook, and seem to be written to direct the attention of the person being tagged to the article.

Suggestions: This is one of those categories where there is not much more to say than the name. Comments containing suggestions.

Questions: Also a category where the name says it all.

Supportive comments: Supportive comments are comments made in defense of someone, including the commentator himself.

Speculative comments: Speculative comments are comments where the commentator is making speculative assumptions, for which there is no real evidence, and making conclusions that cannot reasonably be verified.

Image comments: Sometimes comments contain only an image. Images can contain relevant information, and can even be considered argumentative – a view dating back to classical rhetoric (LaGrandeur 2003, 119).

Links: These are comments that only contain a link to another website. I have not found any of these comments in the data I have analyzed myself, but I’ve made it a category because I did observe these comments while preparing for my research.

Arbitrary comments: Not really a category, just a collection of comments that made me think “WTF??”. Arbitrary comments are comments that are grammatically or contextually difficult to understand.

 

References

Blair, J. Anthony. 2009. “The Rhetoric of Visual Arguments” In Defining Visual Rhetorics, edited by Charles A. Hill and Marguerite Helmers, 41-62. New York: Routledge

Kleining, Gerhard, and Harold Witt. 2000. “The Qualitative Heuristic Approach: A Methodology for Discovery in Psychology and the Social Sciences. Rediscovering the Method of Introspection as an Example” Forum: Qualitative Social Research. 1, no 1, Art 13 – January 2000.

LaGrandeur, Kevin. 2003. “Digital Images and Classical Persusion.” In Eloquent Images: Word and Image in the Age of New Media, edited by Mary E. Hocks and Michelle R. Kendrick, 117-136. Cambridge: MIT Press

Lombard, Matthew, Jennifer Snyder-Duch, and Cheryl Campanella Bracken. 2002. “Content Analysis in Mass Communication: Assessment and Reporting of Intercoder Reliability” Human Communication Research 28, no 4. DOI: 10.1111/j.1468-2958.2002.tb00826.x

Thoughts about Danah Boyd’s Podcast

I was first introduced to Danah Boyd when I read her book “It’s Complicated” about three years ago. I thought it was a great, eye-opening book. Boyd presents a well thought-through argument about the digital lives of teenagers, and introduces perspectives like the fact that kids today (to a higher degree in the US) have fewer opportunities meeting their friends outside of school.

This is one of the topics covered in the podcast, where she also talks about negative aspects of the internet, like bullying and racism. Boyd’s position is that the negative things we hear about on the internet, like cyber-bollying and racist remarks, are not to be blamed on the technology – that they are just a continuation of human behavior in a new medium. While I generally do agree with this, I am interested in how the internet changes the nature of such negative behavior, and if concepts such as anonymity and invisiblity makes it easier for some people to be anti-social online. And this is something that I miss from Boyd’s talk.

But in general, Danah Boyd continues to be an interesting researcher, and the views expressed in her podcast are un-sensational and thought-through.

 

Learning with video games

In the introduction to chapter 4 of Participatory Culture in a Networked Era, “Learning and Literacy”, Mimi Ito writes about “Learning in the wild”. This view of learning argues that the traditional view of learning – that learning involves the passive receiving of information – is outdated. Learning is something that also, and possibly more efficiently, happens in the real world. Ito uses math as an example, and how people can figure out math in real world examples, such as grocery shopping and measuring ingredients.

While I think there is a necessity for traditional learning, I agree with the view presented by Mimi Ito. But I would like to make a case for non-real world, real world learning. By that, I mean the fake worlds of video games. A few weeks ago in class, the subject of learning ethics using the video game “The Walking Dead” came up in discussion. And while I think that this is a great example of using video games in a teaching situation, I feel it barely touches the surface of the possibilities of learning through video games.

But first, I must specify that learning doesn’t have to just be about facts. It’s about building a mental world, with lots of room for facts to be attached to later. As an example, consider the turn based strategy game “Civilization”. The player controls a nation from the dawn of civilization to the modern age. But does that make it a good game for learning history? There are, after all, very little historical facts.

I would argue that a game like civilization, while not presenting a lot of historical facts, allows the player to create a mental image of the history of the world to attach facts to later. First of all, the names of the civilizations and their starting locations, teaches the player about long lost peoples and nations, and their geographical location. Secondly, the technologies the player researches tells a story of the technological development of mankind.

Civilization does not teach history. But when a civilization-player learns about, as an example, the Mongols of the Middle Ages for the first time, he will already have a mental representation of the Mongols in the game: “those purple guys who take over Asia and who are very difficult to have a peaceful relationship with”. This previously unknown people will have already been implemented into the players mental representation of world history.

Mental world-building can happen in a number of games – not just strategy games. And they don’t have to be historically or factually accurate, as long as they create a world for facts and knowledge to be put into.

Mimi Ito writes about participation and learning. Participation, production, collaboration and community organizing can also be vital part of video games in the modern, connected world. People create and learn about architecture through games like Minecraft, and they organize themselves and practice politics through online games like Ark and World of Warcraft.

The Greek and Roman systems of mnemonics focused on creating mental worlds in which to place representations of things that needed to be remembered. With video games, the mental world are built for us. And in stead of being static places, they are vivid, narrative worlds with plenty of space for factual pins where information can be attached and remembered.

 

My Master Thesis: Comments on News Articles

For my master thesis I’m researching comments on news articles and on Facebook. I’m interested in how the social media platform effects the way people talk about subjects in news articles. Currently I’m in the process of fine-tuning my research method, and have come across reliability issues. The research method I’m using is qualitative content analysis. I’m dividing comments into categories, which has turned out to be a surprisingly difficult task. I have yet to find any previous research where comment’s have been categorized into general categories, and I’ve found myself having to conceptualize and define my own categories of comments.

The only thing that I know I will be “making” for my master thesis is a research report. But if there is time, I would also like to create some form of electronic artwork, visualizing any results I might find.

For this semester’s ResNetSem-class I will probably be doing one of two things: 1) write a first draft of my thesis. 2) Finish a part of my thesis, for example a chapter on the history of news article commenting and the spreadability of comments on Facebook.

 

Can news papers create better comment sections through social influence?

It’s an almost established truth that comment sections on news papers are filled with trolls that write hateful and hurtful comments. In my own research, I have been reading dozens of comment sections. And I must say that I don’t recognize this description. In fact, I have seen very few hateful or hurtful comments. This could be because the moderators on the news paper I am studying are doing a good job, or that trolls simply do not comment here.

But of course, I know that trolls are out there and that their commenting is a problem. And while I haven’t seen much of this myself, I have observed a lot of meaningless comments that do not add to the informed discussion that comment sections should be.

So, if we want comment sections to be a place for discussion, sharing of informed opinions and argumentation, what can the news papers do? Moderators is not the answer. They can delete comments that violate community guidelines, but not much else. And you don’t have to brake any guidelines to post a comment that does not contribute to a good discussion.

 

An answer may be found in the importance of social influence, specifically indirect social influence. This is the phenomenon where a person’s opinions and behavior is affected by the information about other people’s action (Cheng et al. 2015, 2). What this means is that people, especially when they are uncertain about a situation, look to other people to find clues about how to act. Cheng et al. found that users of an online bulletin board were affected by indirect social influence. The users conformed to trends in the bulleting board by adopting both positive and negative information.

So how can the effects of indirect social influence be used to better comment sections? In his book, Net Smart, Howard Rheingold writes about virtual community organizers, and compares them to party organizers: “You don’t automatically throw a great party by hiring a room and buying some beer.” (Rheingold 2012, 165). On news paper comment sections, the news papers them selves are the community organizers. And they have “thrown parties”, or created the platforms, by setting up the technical possibility of commenting on articles. But Rheingold writes that the party organizers also needs to “invite an interesting mix of people, greet them at the door, make introductions, start conversations, avert fisticuffs…”

As community organizers, the news papers could potentially better the quality of commenting by being good party organizers. They could set the tone of the discussion by starting the conversation with a well thought-through question to prompt some informed replies in the first comments. And if Cheng’s findings about indirect social influence are correct, this might be enough for later commentators to conform to an established intelligent conversation.

I don’t know if this idea would work, and of course: trolls don’t conform to anything positive (which is where the moderators come in). But I would love to see this at least being tried by a news paper. I’m sure plenty of news papers already start comment sections by writing a generic question like “What do you think?”, or “Are you happy with the election results?”. But I would like to see the journalist ask a more thoughtful question based on unresolved issues in the article, one that forces any potential commentator to stop for a few seconds to think.

Again: I don’t know if it would work, but I think it’s worth trying.

 

References

Cheng Shu-li, Wen-hsien Lin, Frederick Kin Hing Phoa, Jing-shiang Hwang, and Wi-chung Liu. 2015. “Analysing the Unequal Effects of Positive and Negative Information on the Behavior of Users of a Taiwanese On-Line Bulleting Board”. Plos One 10, no. 9. Accessed May 7, 2017. doi:10.1371/journal.pone.0137842

Rheingold, Howard. 2012. Net Smart: How to thrive online. Camridge: MIT Press

Fake News: When Crap Detection Backfires

In chapter 2 of his book, “Net Smarts: How to thrive online”, Howard Rheingold presents us with a few tools to detect crap online, such as incorrect information and false news reports (Rheingold 2012, 78-79). These tools include:

Look for an author. Professional journalists tend to “sign” their articles. Articles that are false, however, tend to be signed by the “News paper” publishing them, or not at all. This makes sense, because most people would not want to sign an article intended to spread false information. But that does not mean that unsigned articles are false – I myself have written technology articles that were not signed by myself, but for whatever reason were published (by myself as editor) without any information about the author.

Look for and evaluate sources. It is important to ask where the author has his or her sources from. If there are none, it does not speak well for the authenticity of the article. But having sources doesn’t automatically mean that the information is correct. Non-serious articles tend to be read by and influenced by like minded individuals who use each other as sources. It reminds me of the “historians” who write about the ancient aliens-hypothesis about aliens visiting ancient cultures, who are all publishing articles and books citing each other’s articles and books in a never-ending circular dance of affirming each other’s points of view.

Look for an owner. If there is no sign of an owner or publisher of the article other than the name of the website itself, there is a good chance the site is not serious. There should also be some contact information. Rheingold mentions the existence of a comment section as a sign of a website being serious. I personally disagree with this point, for two reasons: 1) Plenty of serious websites do not have comment sections, and several serious news websites have over the past few years closed down their comment sections (Ricardo 2014; Clothilde 2016). 2) Plenty of non-serious websites do have comment sections, because this might add to the website traffic.

Look for design flaws and grammatical errors. An amateurish design would indicate a non-serious publisher. The problem with this statement is that it’s becoming increasingly easy for people who know little or no coding to create a professional looking website using, for example, WordPress. I have myself created a news website using WordPress, and it would not be a problem to create one that looks like what one might expect from a professional news organization. Grammatical errors – especially if they are often occurring and obvious – is something that should be taken into consideration when trying to judge the credibility of a website.

Look for what other people think of the website. Who are the people linking to the website? And what are other people online saying about it? I personally find it challenging to judge the validity and seriousness of a website based on what other people think, which is something I will explore in the following section.

Fake News

The most popular election story of 2016 was a false news story.

During the 2016 Presidential Election, the phrase “fake news” became a very popular term. The election saw a new level of false news articles, with 17 of the 20 most popular of them being directed towards the democratic candidate (Silverman 2016). An unprecedented crap detecting operation was undertaken by journalists, and to some degree social media giants such as Facebook (although, not nearly enough). This lead to more awareness surrounding the problem of false news stories, and their potential impact on society – which sounds like a good thing. But is it?

At the time I remember thinking that the awareness of the fake news-phenomenon could easily backfire because of what psychologists have named the Confirmation Bias. This refers to the tendency of people to embrace information that confirms to their established beliefs, while rejecting or ignoring opposing information (Heshmat 2015). This is a purely psychological phenomenon which was discovered long before we became connected to the Internet. So this is how the mind works and process information even before we bring concepts such as the filter bubble and echo chambers into the discussion.

When the term “fake news” became popular, I feared that this would make it easier to fall victim to the effects of confirmation bias. Rejecting and ignoring information that opposes our established beliefs can create a situation of cognitive dissonance, where our actions (believing in X) is contradictory to newly received information (X is wrong). The term “fake news” gives us an easy way to resolve this conflict (the people reporting that X is wrong are lying).

Of course, thinking that people are lying about information that one doesn’t agree with is nothing new. But as the term “fake news” becomes mainstream, it becomes easier to use. And we have seen the effects of this over the past year. The term, which was originally meant to be a description of falsified news stories, have become a standardized response to critical news stories. We see this in President Trump’s response critical journalists, and I have seen it several times during the current Norwegian election – by politicians and voters on both sides.

It is a worrying development. Politicians, and their voters, should be allowed to say that they disagree with news articles, even say that they are wrong. But there is a big difference between “This article is wrong” and “This article is fake”. Fake means wrong by design – an intended action to mislead. And if we start to convince ourselves that some news articles by a news paper are faked, then the whole news paper becomes untrustworthy. And if all we are left with are the few news papers not being critical about the things we believe in, we will be so trapped in our own echo chambers that the very thought of new and opposing information becomes an alien concept. And if that happens, crap detection becomes unnecessary – because we believe that we have already eliminated the crappy news papers – which just so happens to be the ones that we don’t agree with.

 

Sources

Bilton, Ricardo. 2014. “Why some publishers are killing their comment sections” Digiday UK, April 14. Accessed May 8, 2017.http://digiday.com/media/comments-sections/
Goujard, Clothilde. 2016. “Why news websites are closing their comments sections.” Medium, September 8. Accessed May 7, 2017
https://medium.com/global-editors-network/why-news-websites-are-closing-their-comments-sections-ea31139c469d
Heshmat, Shahram. 2015. “What Is Confirmation Bias” Psychology Today, April 23. Accessed September 4, 2017. https://www.psychologytoday.com/blog/science-choice/201504/what-is-confirmation-bias
Rheingold, Howard. 2012. Net Smart: How to thrive online. Camridge: MIT Press
Silverman, Craig. 2016. “This Analysis Shows How Viral Fake Election News Stories Outperformed Real News On Facebook” Buzzfeed News, November 16. Accessed September 4, 2017.                        https://www.buzzfeed.com/craigsilverman/viral-fake-election-news-outperformed-real-news-on-facebook?utm_term=.ciAa8gO1xz#.akxjNlR845

 

Minecraft and Motivation

Minecraft is a revolutionary game in that it changes the rules of motivation and goals. All though there might be exceptions, games have always been about reaching a goal. These goals can be a great many things, like reaching the end of the game or beating a high score, be it your own or someone else’s.

Reaching a goal has also been an important part in many of the various attempts to define games. Jesper Juul claims that goal orientation is the inner layer of his three frames of looking at games. Brian Sutton-Smith sees a game as finite, fixed and goal-oriented, and David Parlett suggests that games consists of the two components ends and means, where ends is the idea that a game is a contest to be won (Egenfeldt-Nielsen, Smith and Tosca 2013, p. 31-37).

The idea of winning a game is usually thought of as reaching the game’s end-state, or finishing the game. This way of thinking is similar to the narrative functions of traditional movies, which are about watching the entire presentation until the end.

The other way we think of winning a game is by performing better than others, or our own previous performances. Both of these ways of winning are dependent on the rules of the game, which define both when the game is over, and how to score points and how many are required to win.

When the player plays a game to reach these external goals that have been set by the game designers, their motivation is external as well. They play the game to beat it, and once the external goals have been reach the game is over (all though, if they really like the game they may choose to play it again).

Minecraft is a game with no obvious external goals. It’s a first-person sandbox-game with no instructions on how to play or what the goal of the game is. Depending on the mode the player plays in, the game is about surviving and building whatever the player wants to build.

Because there are no external goals set in the game one might think that the player would have no motivation to play it. And though it is true that the player might not have any external motivation, playing Minecraft does provide the player with an internal motivation. The player set his or her own internal goals (“I want to build a castle”) which provide internal motivation to play the game in order to reach the player’s goals.

Psychological research has shown that internal motivations for doing something are much stronger than external motivations (Myers, Abell, Kolstad and Sani 2010, p. 171-173). When a person does something for no obvious reason, when there are no external goals or motivations, he needs to justify his actions by creating his own internal motivations. And where an external motivation, like finishing a game by reaching its end-state, disappears when the goal is reached, an internal motivation is long-lasting because an internal goal is both personal and can change to become more complex (as is often the case when playing Minecraft).

What makes Minecraft such a revolutionary game is that it plays on the power of internal motivation for doing something. And because the game is essentially endless our internal goals and motivations can have us playing it for a long time.

This is the end of my essey. So if you will excuse me, I have a castle to build.

 

References

Egenfeldt-Nielsen, Smith and Susana Pajeres Tosca. 2013. Understanding Video Games –  The Essential Introduction. New York: Routledge

Myers, Abell, Kolstad and Fabio Sani. 2010. Social Psychology. Berkshire: McGraw-Hill Education.