Trustworthiness

Valid inquiry in any sphere must

1. Demonstrate its truth value
2. Provide the basis for applying it
3. Allow for external judgments to be made about the consistence of its procedures and the neutrality of its findings or decisions.

The basic issue in relation to trustworthiness is simple: How can an inquirer persuade his or her audiences (including self) that the findings of an inquiry are worth paying attention to, worth taking account of?

Just as a quantitative study cannot be valid unless it is reliable, a qualitative study cannot be transferable unless it is credible, and it cannot be credible unless it is dependable.

Criteria defined from one perspective may not be appropriate for judging actions taken from another perspective.

---

Credibility — Does it “ring true”?

(Is there compatibility between the constructed realities that exist in the minds of the inquiry’s respondents and those that are attributed to them?)

1. Prolonged Engagement (Stay in the field until data saturation occurs.)

a. counters distortions from researcher’s impact on the context
b. limits researcher biases
c. compensates for effects of unusual or seasonal events

2. Persistent Observations (Consistently pursue interpretations in different ways in conjunction with a process of constant and tentative analysis. Look for multiple influences. Search for what counts and what doesn’t count.)

3. Triangulation (The best way to elicit the various and divergent constructions of reality that exist within the context of a study is to collect information about different events and relationships from different points of view.)

a. ask different questions
b. seek different sources
c. utilize different methods

4. Referential adequacy (What materials are available to document your findings? Video tape provides a good record but it can be obtrusive.)

5. Peer Debriefing (This is done with a similar status colleague (not with a junior or senior peer) who is outside the context of the study and who has a general understanding of the nature of the study and with whom you can review perceptions, insights, and analyses.)

a. provides a “devils advocate”
b. tests working hypotheses
c. helps develop next step
d. serves as a catharsis

6. Member Checks (Go to the source of the information and check both the data and the interpretation.)

a. assesses intentionality of respondents
b. corrects errors
c. provides additional volunteer information
d. puts respondent on record
e. creates an opportunity to summarize which is the first step to data analysis
f. assesses the overall adequacy of the data in addition to individual data points

Transferability
(extent to which the findings can be applied in other contexts or with other respondents)

Most contemporary researchers view applicability in terms of generalizability and address the issue by focusing on those aspects of the inquiry that do not shift. The naturalistic researcher maintains that no true generalization is really possible; all observations are defined by the specific contexts in which they occur.

The naturalistic researcher does not maintain that knowledge gained from one context will have relevance for other contexts or for the same context in another time frame. In a traditional study it is the obligation of the researcher to ensure that findings can be generalized to the population; in a naturalistic study the obligation for demonstrating transferability belongs to those who would apply it to the receiving context (the reader of the study).

Strategies for Transferability

1. Thick Description
Because transferability is a naturalistic study depends on similarities between sending and receiving contexts, the researcher collects sufficiently detailed descriptions of data in context and reports them with sufficient detail and precision to allow judgments about transferability to be made by the reader.

2. Purposive Sampling
In contrast to random sampling that is usually done in a traditional study to gain a representative picture through aggregated qualities, naturalistic research seeks to maximize the range of specific information that can be obtained from and about that context by purposely selecting locations and informants that differ.

Dependability
An inquiry must also provide its audience with evidence that if it were replicated with the same or similar respondents (subjects) in the same (or a similar) context, its finding would be repeated.

1. Don’t Need to Do It
Since there can be no validity without reliability (and thus no credibility without dependability), a demonstration of the former is sufficient to establish the latter. If it is possible using the techniques outlines in relation to credibility to show that a study has that quality, it ought not to be necessary to demonstrate dependability separately. (Arguable)

2. Overlap
In effect, overlap methods represent triangulation which is typically undertaken to establish validity, not reliability, although demonstration of the former is equivalent to demonstration of the latter. (Still Arguable)

3. Stepwise Replication
Teams deal with data sources separately and, in effect, conduct their inquiries independently. (Not recommended)

4. Inquiry Audit
An auditor examines documentation (through critical incidents, documents, and interview notes) and a running account of the process (such as the investigator’s daily journal) of the inquiry. The auditor examines the process of the inquiry, and in determining its acceptability the auditor attests to the dependability of the inquiry. The inquiry auditor also examines the product–the data, findings, interpretations, and recommendations–and attests that it is supported by data and is internally coherent so that the “bottom line” may be accepted. This latter process establishes the confirmability of the inquiry. Thus a single audit, properly managed, can be used to determine dependability and confirmability simultaneously.

…..Lincoln, Y. S., & Guba, E. G. (1985). Naturalistic inquiry. Newbury Park, CA: Sage.

Confirmability
This is the degree to which the findings are the product of the focus of the inquiry and not of the biases of the researcher

Confirmability Audit Trail
An adequate trail should be left to enable the auditor to determine if the conclusions, interpretations, and recommendations can be traced to their sources and if they are supported by the inquiry.

Lincoln and Guba (1985) noted that Halpern (1983) suggested six classes of raw record data to be reviewed.

1. Raw Data
recorded videotapes, written field notes, documents, survey results

2. Data Reduction and Analysis Products
write-ups of field notes, summaries and condensed notes, theoretical notes such as working hypotheses, concepts, and hunches

3. Data Reconstruction and Synthesis Products
themes that were developed, findings and conclusions, final report

4. Process Notes
methodological notes, trustworthiness notes, audit trail notes

5. Material Relating to Intentions and Dispositions
inquiry proposal, personal notes, expectations

6. Instrument Development Information
pilots, forms and preliminary schedules, observation formats, surveys

A more detailed account of conducting an audit can be found in Lincoln, Y. S., & Guba, E. G. (1985). Naturalistic inquiry. Newbury Park, CA: Sage.

Del Siegle, PhD
del.siegle@uconn.edu