Rewriting notions of ‘validity’ through research in the postmodern
Validity and James Scheurich’s reflexivity - research upon research as a social practice
Validity, in general terms, refers to the appropriateness of a tool for the job. A hammer is appropriate for knocking nails into wood, where a screwdriver is not appropriate (such ‘self evident’ general validity is usually referred to as ‘face validity’). A research instrument that measures, describes or evaluates what it sets out to measure, describe or evaluate is valid. Validity however takes a number of forms. Cohen, Manion and Morrison (2000, p.105) suggest that ‘in quantitative data validity might be improved through careful sampling, appropriate instrumentation and appropriate statistical treatments of the data’; where ‘in qualitative data validity might be addressed through the honesty, depth, richness and scope of the data achieved, the participants approached, the extent of triangulation and the disinterestedness or objectivity of the researcher.’
Here, the authors offer a perfect lead for a postmodern reworking of notions of validity. Just what might Cohen et al mean by qualifiers such as ‘careful’ and ‘appropriate’? Notions such as ‘honesty, depth, richness and scope’ are problematic, and to equate validity with ‘the disinterestedness or objectivity of the researcher’ places their definition firmly within a positivist tradition, neglecting even the interpretivists’ view of the role of researcher, which is firmly criticised as never being neutral, especially in areas such as ethnographic research. Action researchers would also see such a definition as limiting research to an activity done on people by a disinterested researcher, rather than with people, for emancipatory or practice change reasons – research that Lather (1986) has referred to as ‘openly ideological’. Winter (2000, p.2) notes that any one particular definition of validity is ‘entirely relative to the … belief system from which it stems’, describing the methodological persuasion of the researcher as the key issue in validity arguments and applications.
Cohen et al (2000, p.105) accept that validity ‘should be seen as a matter of degree rather than an absolute state’, but frame validity as an ideal research should strive for. The aim is reduction in contamination of data collection and analysis such as bias and standard error: ‘at best we strive to minimize invalidity and maximise validity.’ Invalidity is demonised: ‘it is both insidious and pernicious as it can enter at every stage of a piece of research’ (Cohen et al, 2000, p.115). Foucault’s (1989) work on the classification ‘explosion’ during the Enlightenment reminds us of one of the typical mechanisms by which a normative and regulative social discourse exerts its power, through classifications and diagnostic schemata (see also Eco, 1999). The more one multiplies up the classificatory descriptors, the greater the web of regulative power, as productivity of identities and meanings is increased. Cohen et al (2000, pp.104-105) list eighteen kinds of validity (content, criterion-related, construct, internal, external, concurrent, face, predictive, consequential, systemic, catalytic, ecological, cultural, descriptive, interpretive, theoretical, and evaluative), faithful to eight ‘positivist principles’: controllability, replicability, predictability, the derivation of laws and universal statements of behaviour, context-freedom, fragmentation and atomization of research, randomization of samples, and observability.
Read Cohen, Manion and Morrison (2000) pp. 104-105
Scheurich (1997, p.80) offers a postmodern deconstructive investigation of such a proliferation of kinds of validity: ‘the myriad kinds of validity are simply masks that conceal a profound and disturbing sameness.’ In what is an example of a piece of textual research based loosely around a discourse evaluation, Scheurich sets out to define this ‘singularity of purpose and function’, even where it is claimed that validities arise from differing epistemological orientations.
Validity originates as a truth claim, and postmodernism is sceptical of such claims (Gaskell and Bauer, 2000). Modernist research methodology sees reliability (largely replicability) as a necessary but not sufficient condition for the rigour of research, where validity provides sufficiency. Another way to put this is that within the positivist model, invalidity invites distortion of possible truth or fact, acting as a corruption to objectivity, and leaving the researcher as untrustworthy guide to truth and reality, as Cohen et al (2000) vigorously claim above. In Foucauldian terms, discourses of validity offer a policing of truth claims. Validity equates with legitimacy, and legitimacy is relative to the standpoint of the particular dominant epistemology (theory of knowledge).
Scheurich puzzles as to why postpositivists or interpretivists, having challenged the scientific rationalism of positivist research and given value to subjectivity, descriptive account, and the local rather than the universal, would want to carry over to their methodological concerns the obsession with validity that positivists demonstrate. While early postpositivist work used conventional science criteria and would be expected to still be tied to conventions of validity, new paradigms have emerged, such as feminist research, where validity has not been fundamentally questioned, but reinterpreted within the new paradigms and forms. Patti Lather (1986, 1991, 1993) has been the leading voice in this field.
Scheurich calls positivist validities ‘originary’ and postpositivist validities ‘successor’. The shift may be characterised as one from validity as a truth and trustworthiness claim based on transhistorical and transcultural universal principles (‘absolutism’ and the scientific method), to validity as a relative truth and trustworthiness claim based on the consensus achieved within a community of practice. Here, validity is socially constructed and situated (historically and culturally contingent). Scheurich’s point is that in each case, the kinds of validity generated are ‘masks’ in the sense that, at the end of the day there is only one principle at stake in both ‘originary’ and ‘successor’ validities: validity is a regulative device separating trustworthiness from untrustworthiness.
Patti Lather (1986) maps four validity methods: triangulation, construct validity, face validity and catalytic validity, in an attempt to offer a ground for ‘trustworthiness’ in politically oriented, emancipatory research. Catalytic validity is central to such research, concerned with the degree to which participants are genuinely empowered or raise consciousness through the research process. This offers the rigour and systematising that such research models fear a lack of will lead to them being judged as untrustworthy (invalid) by conventional researchers. In terms of psychoanalytic defence mechanisms, this could be seen as a paranoid reaction or rationalization. As Scheurich points out, Lather simply redraws the lines of trustworthiness. For such emancipatory research, lack of attention to empowerment automatically invalidates the research (which may then be labelled as ‘oppressive’), reducing validity to a policing or regulatory mechanism.
In Scheurich’s ‘deconstructive investigation’ of validity, ‘governmentality’ (Foucault’s term, appropriated to describe the regulatory function of research through validity checks) converges with the Derridean notion that such regulation operates through the maintenance of a binary: inclusion/exclusion as trustworthy/untrustworthy research, whatever the research paradigm. Postmodernism’s project is to dissolve such modernist binary thinking (intrinsic to Western knowledge construction since Plato) to offer a plurality and polyvocality. Research has been the engine by which the knowledge machine is governed or legitimated, converting the ‘other’ of the researched into the ‘self/same’ of the researcher and chosen (dominant) paradigm, where validity has acted as the regulatory principle through which such a conversion is legitimated. The conversion of the ‘raw’ other (the researched) into the ‘cooked’ self/same (the researcher and chosen paradigm) is a validation process (we talk of ‘raw data’ prior to its manipulation, interrogation, analysis and synthesis) – hence Nietzsche’s insight, developed by Foucault, that the ‘will to knowledge’ is equivalent to the ‘will to power’. In psychoanalytic terms, such a conversion process, as a defence mechanism, is termed ‘incorporation’. It is based on the principle of ‘eat or be eaten’. Scheurich (1997, p.86) quotes Helene Cixous’ description of this process as ‘a politics of arrogation’. Scheurich views it as ‘a totalizing system’ that is ‘the academic version of Western imperialism’. He coins the term ‘imperial validity’ to describe this controlling and legitimising process in research.
Up to now, Scheurich has offered a reflexive research upon research, exposing an unacknowledged (and necessarily unconscious) contradiction at the heart of validity claims across modernist research paradigms and glossing this contradiction with insights drawn from Foucauldian models of regulation and legitimacy. However, Scheurich (1997, p.88) does have an alternative suggestion as a consequence of this deconstructive inquiry. He wishes to shift from a ‘two-sided map’ that is ‘highly problematic’, to a ‘play of difference’, that rejects dualism for a celebration of ‘polyphony, multiplicity’. Such ‘new imaginaries of validity’ need to ‘unmask and undermine’ the dualism pervading current models that offers a regulatory mechanism. Validity notions in educational research need to be both deconstructed and reconstructed.
We have seen that new validities cannot simply be invented to accommodate new research paradigms. This merely reinforces the oppressive self/same:other duality. What we might do, suggests Scheurich (1997, p.88) is to pay more attention to ‘dialogue and collaboration between the researcher and the Other’. However, this requires extraordinary sensitivity to ‘local knowing, local validity, and local choices’, avoiding ‘any non-local, prescriptive meta-narrative’. To extend Scheurich’s analysis, such sensitivity is not necessarily embedded in the chosen research methodology, but is an attributional state of researchers, accepting that such identities are themselves potentially constructed through the exercise of the discourses of ‘research’. Important attributes include ‘attentive care’, a version of the ‘care’ as opposed to ‘justice’ ethic proposed by feminists (particularly Carol Gilligan) as an important factor in the ethics of research.
The researcher values tolerance of difference and ambiguity, attentive listening rather than prescriptive interruption, elasticity in method to adapt to local concerns, and a willing suspension of habitual practices to explore emergent meanings. Sometimes, this may offer a ‘hard’ edge, where dissensus must be tolerated rather than striving for consensus, and unknowing may be a preferred state of being to knowing. Importantly, a literacy of ‘difference’ is activated. Scheurich (1997, p.89) describes this ideal attitudinal state informing research in the postmodern as ‘holding open a space of difference’. It is both an ethical and aesthetic task, modelled by postmodern ethnography that resists the temptation to revert to a ‘civilizational project’.
Scheurich notes that Lather has moved on from her (1986) previous remarks on validity, set in an emancipatory paradigm, to a new formulation (1993) arising from a feminist poststructuralist perspective. She describes ‘ironic’, ‘paralogical’, ‘rhizomatic’ and ‘voluptuous’ validities under one general heading: ‘transgressive validities’. While still generating classifications, Scheurich sees Lather as offering an alternative approach to validity as a regulatory device for trustworthiness. Each kind of validity is specific to particular research contexts, but Scheurich’s reading suggests that ‘transgressive validity’ as a generic category has three foci:
a critical interest in destabilising truth claims
a postmodern focus in generating new literacies appropriate to local contexts and emergent meanings
a political agenda that brings a relativist and context-specific ethics to each research occasion, promoting critical reflexivity.
Scheurich concludes his research article on validities with a pragmatic yet pessimistic note, realising how difficult the project of activating ‘transgressive validity’ is, and how it also may transform from ideal to ideology. Such a postmodern challenge to validity is a project, a direction, not a goal for mastery. He sees the ‘carnival of the play of difference’ at work in our culture. Researching the carnival must be an act of dialogue with, not monologue about, such difference.
Cohen, L., Manion, L. and Morrison, K. (2000, 5th ed.) Research Methods in Education. London: Routledge/Falmer.
Eco, U. (1999) Kant and the Platypus: Essays on Language and Cognition. London: Secker & Warburg.
Foucault, M. (1989) The Order of Things. London: Routledge.
Gaskell, G. and Bauer, M.W. (2000) Towards Public Accountability: beyond Sampling, Reliability and Validity. In M.W. Bauer, and G. Gaskell (eds) (2000) Qualitative Researching with Text, Image and Sound. London: Sage.
Lather, P. (1986) ‘Issues of validity in openly ideological research: Between a rock and a soft place’, Interchange, 17:4, pp. 63-84.
Lather, P. (1991) Getting Smart: Feminist Research and Pedagogy With/In the Postmodern. London: Routledge.
Lather, P. (1993) ‘Fertile obsession: Validity after poststructuralism’, The Sociological Quarterly, 34:4, pp. 673-93.
Scheurich, J.J. (1997) Research Method in the Postmodern. London: The Falmer Press.
Winter, G. (2000) A Comparative Discussion of the Notion of ‘Validity’ in Qualitative and Quantitative Research. The Qualitative Report. 4:3/4. http://www.nova.edu/ssss/QR/QR4-3/winter.html