The below is written in response to this Guardian article and thread: "Status affects how readily people return smiles, research reveals - People who feel powerful are more likely to return smiles of those they see as low status, according to study"
Disappointing, Alok Jha, you should really have known better before splashing this "scientific" research on the pages of the Graun.
Fundamental flaws of the study as reported:
1. Inappropriate Indicators for both Independent and Dependent variables
The stimulus is meant to be "status", the response "submission", and the mediator "power".
Never mind that there is a question about whether a "smile" could be an indicator of "submission" as other commenters already picked up on. And never mind whether the electro signals picked up by the new-fangled gadget are actually corresponding to a smile, as even Carr admits that the subjects were just sat there with a stoic face watching a video screen. (And how can you justify that the signals recorded are indeed a submissive "smile" and not an arrogant "smirk"??? Such question is clearly too complicated for Carr and his team to consider and so they conveniently ignore that and carried on regardless.)
Never mind also that "Status" is indicated by using the proxy of "professsionals" versus "menial workers". As someone pointed out earlier, without knowing the subjects' own background and history it's hard to judge whether they would actually perceive these actors-in-costumes-on-a-TV-screen are indeed high versus low status (and indeed, whether their status perceptions are to do with these actors qua actors, or the roles that they were assigned to play). It's just too bad Carr and his team didn't think about using some kind of standardised measurement tool regarding individual perceptions of occupational status, to make sure they are measuring what they think they are measuring as regards participants' own, rather than researcher-assumed, status perceptions.
Never mind also that "Power" is "primed" by asking subjects to write an essay about their positive and negative experiences. That's the most egregious of the indicators used of the three basic variables being tested in this experiment. Carr and his team chose to ignore validated tools that have been developed to measure self-efficacy (which is really a more appropriate concept in this scenario than the multidimensional construct "Power"), using their laughable exercise as a proxy to gauge subjects' self-perception of "power" when nothing had actually been used to measure their perceived power (or self-efficacy).
It was all just assumed that writing an essay about yourself will have primed you to feel powerful or disempowered when confronting strangers-on-a-TV-screen, regardless of what personality type you are (and indeed, whether you like watching TV or not!).
No, never mind ALL OF THE ABOVE, because they aren't even the worst flaw of this study design. Even if one takes all of the above on face value (no matter how straining of the face validity test these indicators are presenting to us), I cannot believe that no-one, but no-one (not the ethics committee, not the funding body, and certainly not Carr nor his helpers), stop for one second and ask themselves this:
Why is it okay to assume that WATCHING A VIDEO of some actors on a TV screen is the SAME AS INTERACTING WITH THEM IN PERSON????
Helloooo?? Do you think you are actually interacting with George Clooney (or say, Scarlett Johannsson) when you watch him/her on screen??? Somehow, I'd think one would have to be pretty deluded to think this way.
I think the 55 subjects of this study could pretty much tell the difference between smiling at someONE in person (whether initiating or returning a smile), or smiling AT someTHING that happens on a TV screen.
I don't think these 55 subjects, if they are halfway normal humans, would be smiling PERSONALLY AT, say, Zac Braff on TV, irrespective of whether Braff was playing a doctor in Scrubs or a bum in Garden State.
Sure, they might smile at the show, but that's not the same thing as initiating or returning a PERSONAL smile to the actor now is it??
So why is Alok Jha bringing this excretable piece of "research" to our attention??
2. Inappropriate and Unjustified Overreaching of Conclusion and Generalisability
Recall that this is a lab experiment with 55 volunteers. No actual demographics of the volunteers given, and given the sample size, impossible to generalise their facial expressions as being representive of the human race (or even just the American population in general). Yet Carr glibly talks about his findings as if they are automatically applicable to everyday situatoin that you and I would face in our normal course of life.
It is even cheekier for him to use the Senior VP in an office as an example, when he knows full well that his under-powered lab study did not take place in a natural setting whatsoever.
But then, that's what loads of I/O psychologists do - they think they are generating "scientific truth" with an experimental design using fancy gadgets, when really they failed abysmally to even think up a half-decent research design to allow them to test what they wish they are testing.
If anthropologists try to make claims like this Carr guy did given similar study sample and design, they will be laughed out of the auditorium much less having their findings published in national newspapers because their findings happen to be "cute" and "counter-intuitive"!
Really, Alok, you did science?
--------
This was first published on 15 October 2012 at 18:57. It was submitted to the Guardian thread on my moderated account just a couple of minutes prior to this time-stamp.
Unfortunately, as of 16 October 16:48, the mods don't have the balls to put the above criticism through. Despite the Guardian always claiming they want to publish "good" rather than BAD SCIENCE, and despite the campaign run by one of its own star columnists Ben Goldacre who is a lot harsher on bad science reporting than I had done above.
Oh well, Guardian is proving itself to be quite equal to my low estimation of it.
One more thing I will say: it is a disgrace to get a grad with only a primary degree in physics to act as "Science correspondent", much less to cover social science studies! Alok Jha is clearly out of his depth and in over his head. It's like trying to get a reporter just learning his chops on Russia to become a seasoned Middle-East correspondent, FFS.
No comments:
Post a Comment