Ranking forensic journals through content analysis

Illustration credit: Jock Alexander, The Australian

You have no doubt heard of journal "rankings." A journal's rank conveys information about the impact and quality of a journal. This can be useful information for both authors and consumers. An author might want to consider a journal's prestige, and the difficulty getting published in it. For consumers -- including expert witnesses who might be relying on a particular article in court -- ranking can serve as a proxy for the accuracy and reliability of a journal's content. How much should the trier of fact trust the information in this journal?

But there are lots of methods for ranking journals -- the Impact Factor, the Eigenfactor, the h-index, just to name a few. And with the proliferation of journals in forensic psychology, it gets hard to keep track. Which journals have the best reputations? Which are the most cited? Which provide the broadest coverage of forensic psychology topics?

One popular way to rank-order journals is based on reference counts. How many times a journal is cited is an indicator of its reputation. In forensic psychology, according to an unpublished study by S. Black, the top-referenced journals are (in rank order):

1. Law and Human Behavior
2. Behavioral Sciences and the Law
3. British Journal of Psychiatry
4. Journal of Forensic Psychiatry and Psychology
5. American Journal of Psychiatry
6. Criminal Justice and Behavior

Now, a researcher with training in both psychology and library science has taken a somewhat different approach, devising a clever content-analysis procedure to rank-order journals in our field.

Chris Piotrowski started by screening several texts in the field and choosing terms that are popular both in research and practice. The 16 terms were: eyewitness testimony, competency to stand trial, alcoholic blackouts, infanticide, sentencing, forensic evaluations, polygraph, malingering, jury selection, homicide, diminished capacity, insanity defense, child abuse, Daubert standard, child custody and expert witness.

Next he used PsycNET, "the recognized major bibliographic resource in the social and behavioral sciences that indexes scholarly and professional journals," to run keyword searches on his 16 terms. For each search term, he rank-ordered journals based on the frequency of hits; a journal's total ranking was obtained by summing across all 16 terms.

The winners were (in rank order):

1. Journal of the American Academy of Psychiatry and Law
2. PsycCRITIQUES (formerly, Contemporary Psychology)
3. Law and Human Behavior
4. Behavioral Sciences and the Law
5. American Journal of Forensic Psychology
6. Journal of Psychiatry and Law
7. Bulletin of the American Academy of Psychiatry and Law
8. Mental & Physical Disability Law Reporter
9. American Journal of Psychiatry
10. American Psychologist
11. Journal of Forensic Psychology Practice
12. International Journal of Psychiatry and Law
13. Journal of Criminal Justice
14. Professional Psychology: Research and Practice
15. Journal of Applied Psychology
16. Archives of Clinical Neuropsychology
17. Psychological Reports
18. British Journal of Psychology
19. Psychology, Public Policy, and Law

I would be a little cautious about relying on this method, because the choice of keywords -- which is open to manipulation -- might influence the rankings. But as you can see, there is overlap between this method and the more traditional citation-count method used by Black. For instance, Law and Human Behavior and Behavioral Sciences and the Law made it into the top four, no matter which method was used. There are some noticeable differences as well, with several journals that were highly cited in Black's study not ranking high using this content analysis method.

For more information on the method and the exact scores for each journal, feel free to contact Dr. Piotrowski (HERE); I'm sure he will be happy to share a copy of the article, which is published in the current issue of the American Journal of Forensic Psychology.

Oh, in case you were wondering, that journal is number five on his list.

The article is: Top cited journals in forensic psychology: An analysis of the psychological literature (2012), American Journal of Forensic Psychology 30 (2), 29-38.