One interesting wrinkle in the discussion (here) about whether to pursue a standard OLS regression or multi-level model strategy involves the influence of sample size, cell counts, and the ratio of predictors to cases.
A very cool user-written program ("maptile") allows Stata users to generate maps that present state-, county- or zip-code-level data. For a list-serv discussion of (and links to) the mapping software click here. The mapping software even has its own website with additional information and links (here). I just completed downloading the software and, within minutes, was able to replicate (below) a U.S. map of percent Urban population (drawn from 1980 Census). Those who work with state data might find this mapping software helpful.
Responding to growing public criticism of American military leaders' treatment of sexual assault cases, Congress re-wrote key parts of the Uniform Code of Military Justice ("UCMJ"). Motivating many of Congress' reforms was a belief that "commanders are predisposed to believe the offenders, who are often good soldiers, and to disbelieve the victims, who may have behaved in ways that commanders dislike." If so, such a bias would imply "high levels of attrition as the commanders undervalue the [sexual assault] cases and divert them away from the legal process."
Eric Carpenter (FIU), whose prior service includes stints as an Army helicopter pilot and in the JAG Corps, sets out to test underlying assumptions behind the UCMJ reforms in a recent paper, An Empirical Look at Commander Bias in Sexual Assault Cases. Key findings are described in the excerpted abstract.
"This study ... measures the attrition of sexual assault cases in the precise phase of the case processing that the commanders control – the decision to take action in the case. Using data that I received from the Army through the Freedom of Information Act, this study measures how commanders disposed of every founded sexual assault and sexual contact offense in the Army from 2008-2011. Further, this study tests the counterfactual – how commanders treated other similar cases: homicides, robberies, and assaults.
This study finds that commanders treat non-penetrative sexual assaults the same or more seriously than they treat simple assaults. Further, when commanders decide to take action on penetrative sexual assault cases, commanders send more of those cases to trial than they do with comparable crimes. However, commanders decide to take no action in penetrative sexual assault cases more frequently than they do with other comparable crimes. This study includes a secondary finding that commanders treat domestic violence cases more seriously than they treat other similar assault cases.
The results of this study should inform the national debate on whether Congress should remove commanders from the process. These results suggest that step may not be necessary, and research and reform efforts should focus on how military law enforcement handles these cases."
A call for papers (excerpted below) for a 2017 conference, hosted by Academica Sinica in Taiwan and co-sponsored by the Society for Empirical Legal Studies (SELS), further evidences empirical legal studies' widening global footprint and reach. KC Huang (a JELS international adviser) helped found Academia Sinica's Empirical Legal Studies Center. Now that KC is a member of Taiwan's Congress, Yun-chien Chang directs the Center and organizes CELSA. The conference's host, Academia Sinica, has already organized three international conferences and multiple ELS workshops, making Taipei a critical hub of ELS activity in East Asia.
"The first Conference on Empirical Legal Studies in Asia (CELSA) brings together, in Asia, researchers from law and other fields who seek to develop empirical accounts of law and legal institutions.
Papers are selected through a rigorous peer review process. CELSA will consider empirical papers across all areas of law. Quantitative data analysis is preferred. Only submissions in English are considered. While authors are strongly encouraged to submit works-in-progress, submissions should be completed drafts that present main findings. Submitted papers must be unpublished (and expected to be unpublished at the time of the conference). If accepted, authors will have an opportunity to submit a revised draft prior to the conference. Each paper will be assigned a discussant and authors of accepted papers are expected to be willing to act as discussants of another paper."
CELSA is set for June 13-15, 2017 (Tues.-Thurs.) at Taiwan's Academica Sinica.
Call for papers submission deadline: 15 February 2017.
Click here for more submission and conference information; please direct specific questions to Prof. Yun-chien Chang, at: firstname.lastname@example.org
While the bankruptcy code may not strike many as particularly "glamorous," as Bob Lawless (Illinois) notes, it involves a segment of the court system that will oversee "780,000 cases this year." And "glamor" aside, good empirical work on this topic is, according to Lawless, "exactly the sort of work we could use more of."
Cracking the Empirical Code: An Empirical Analysis of Consumer Bankruptcy Outcomes, by Sara Greene (Duke) et al., is a recent addition to a growing list of promising empirical work in bankruptcy. The paper draws on a randomized national sample of bankruptcy filings in 2007 from the Automated Access to Court Electronic Records (AACER) system. The paper's abstract, excerpted below, summarizes the key findings.
"... Despite the accolades from policymakers, the hard truth is that the majority of the half-million families each year that seek refuge in chapter 13 bankruptcy will not achieve the debt relief of a discharge. Prior research found that those who drop out of bankruptcy quickly endure the serious financial struggles that they had before bankruptcy — now even worse off for having spent thousands of dollars to seek help. Despite the profound inefficiency of chapter 13 bankruptcy, we previously did not know what differentiates those who succeed in chapter 13 from those who fail. This article is the first study to use a national random sample to predict which debtors obtain a discharge of debt. Using sophisticated statistical techniques that allow us to control for unobservable or unmeasurable effects at the local level, we identify the factors that make completing chapter 13 bankruptcy more likely. We find, among other robust effects, that blacks are more than twice as unlikely to receive debt relief than non-blacks, that those without an attorney have extremely low odds compared to those who hire an attorney, and that families with children fare worse. We also find that the local variations in bankruptcy practice that have been deemed “best practices” do not correlate with higher rates of bankruptcy completion. We discuss the implications of our findings for the millions of families who struggle to repay their debts in bankruptcy, and suggest concrete fixes to increase the efficacy of the consumer bankruptcy system. This article upsets the debate about bankruptcy reform and will help shape policy and practice in upcoming decades."
While graphics in Stata can be a pain, sometimes quick coding suggestions can help modify traditionally "standard" presentations in small ways that add clarity and, thus, value. For example, those with experience with quartile-box plots might appreciate a discussion (here) about the code needed to add marker labels indicating the value scores at the 25th, 50th, and 75th percentiles. (The example below illustrates this modification.)
While the current (or, candidly, any) replication crisis should give all empirical scholars pause, much of the attention (rightly or wrongly) focuses on psychology (and, by implication in the ELS world, law & psychology). While what explains the current focus on psychology is not entirely clear, in a blog post (here) Andrew Gelman (Columbia--statistics) offers a few reasons. One quick take-away, however, is that at least some of Gelman's reasons are neither necessarily nor structurally limited to psychology. Moreover, the first sentence in Gelman's post underscores another important point: The replication crisis is a "big deal" and one that warrants careful and sustained attention.
A while back, we posted information on a proposed idea at Harvard Law School that was seeking to "staff-up" (here). I am delighted to report that the "idea" has now launched as an up-and-running program--Harvard Law School's Access to Justice Lab. According to Research Director Chris Griffin, the Lab, affiliated with HLS's Center on the Legal Profession, is a "venture for developing evidence-based solutions to problems of court administration and legal services provision. Our goal is to understand, using randomized control trials, what methods truly enhance engagement with and efficiency in the system." The Lab's mission includes: "combat[ing] resistance among the bench and bar to rigorous empirical thinking about the U.S. legal system, and to study it using the gold standard of randomized field experiments." Those interested in learning more about the Lab and its work should visit its website, follow on Facebook or Twitter, or subscribe to the blog.
Conventional wisdom--and many legislative histories--imply that seat belt laws help to reduce automobile accident-related deaths. In a recent paper, Do Seat Belt Law Still Work? Replication and Re-Evaluation of Recent Evidence, Sam Harper (McGill) and Erin Strumpf (McGill) set out to replicate, refine, and extend existing empirical work. Drawing on US traffic fatality data from 2001-10, the authors findings' do not support conventional wisdom. The paper's abstract follows.
"Recent work suggests that mandatory seatbelt laws are associated with lower motor vehicle accident mortality, but it is unclear whether this is due to increased enforcement. We study this association using 2001-2010 US data on traffic fatalities among individuals ages 10 and over and we test the robustness of unstated assumptions by replicating and extending prior work. Using Poisson regression adjusted for state-level traffic safety policies (seat belt laws, blood alcohol laws, speed limits), miles traveled, and median income, we replicate the covariate-adjusted association seen in prior work (Rate Ratio [RR]=0.80, 95%CI 0.74,0.87). Further adjustment for state and time fixed effects showed no evidence of a causal effect (RR=1.00, 95%CI 0.95,1.06). We find little evidence that seatbelt laws reduce traffic fatalities, and prior work was confounded by general improvements in other environmental determinants of motor vehicle accident mortality. Replications studies have the potential to increase the integrity of research findings."
In an interesting move, the Journal of Law and Courts is "opening its pages to most types of replications, successful or unsuccessful." In its Call For Replications, JLC's editors recognize that journals' traditional "emphasis on novelty means that a replication study has almost no chance of being published unless the replication fails and the failure can be presented as persuasive evidence for an alternative theory." While excluding "successful direct replications, employing the same data and models as the original study," the editors welcome any study that asks whether "findings in one domain extend to another domain or can be replicated using different measures or models."
Social security claimants' experiences appealing claims can vary considerably across U.S. district courts. Drawing from a dataset of more than 20,000 appeals, Jonah B. Gelbach (Penn.) and David Marcus (Ariz.), in A Study of Social Security Disability Litigation in the Federal Courts, find, in part, that: "District judges in Brooklyn decide these cases pursuant to one set of procedural rules and have in recent years remanded about seventy percent to the agency. Magistrate judges in Little Rock handle this docket with a different set of rules and have in recent years remanded only twenty percent." An excerpted abstract follows.
"Our report makes four contributions. The first is a thorough introduction to the process by which a disability benefits claim proceeds from initial filing to a federal judge’s chambers. This description is intended to deepen understandings of where many of federal civil cases come from, and why they raise the same sorts of concerns repeatedly.
Second, the report provides some context for understanding why the federal courts remand claims to the agency at the rate that they do. We argue that the federal courts and the agency have different institutional goals, commitments, and resources. These differences would cause a sizable number of remands even if the agency adjudicated claims successfully and the federal courts applied the appropriate standard of review. Third, we undertake extensive statistical analysis to try to understand what factors explain the sharp variation in district-level remand rates. Circuit boundaries account for some, but not all, of this disparity. After excluding a number of other potential causes, we hypothesize that district courts remand claims to the agency at different rates in part because uneven adjudication within the agency produces pools of appeals of differing quality. Finally, the report analyzes contrasting procedural rules used by different districts to govern social security litigation. We argue that these differences are unnecessary and create needless inefficiencies. We conclude with a set of recommendations to improve social security litigation within the federal courts."
A young up-and-coming empirical scholar at Hebrew University, Netta Barak-Corren, passed along her summary of a recent paper that considers such questions as: How do social diversity's negative effects distribute? What mechanisms help explain the relationship between diversity and discriminatory behaviors and why do they vary in prevalence and strength across locations? Netta's post follows.
“Folks working on issues related to discrimination and bias might find interest in a recent paper that studied the impact of local context on patterns of discrimination. In Intergroup Behavioral Strategies as Contextually Determined: Experimental Evidence from Israel, Ryan D. Enos and Noam Gidron (Harvard) report the results of multisite lab-in-the-field experiments and survey responses collected across 20 locations. Participants played dictator and public good games against in-group and out-group members.
The researchers found large variation in the extent of discriminatory behavior conditioned on the level of diversity and segregation at each location they studied. The methodological conclusions are particularly interesting. Enos and Gidron demonstrate how 'different conclusions would be reached by locating a study in different locations, even within the same country.' They emphasize the implications for research design: 'Because behavior is a function of context as well as of the players, scholars may observe the operation of different mechanisms depending on the location chosen for research.' For example, they show that researchers may study the same behavior (in-group bias), use the same measures (dictator and public good games), and recruit participants from the same populations and still get substantially different results depending on the location of their lab. Comparing pairs of cities with similar population proportions but different levels of segregation they show that a study conducted in the more segregated city would yield stronger results of in-group bias than the same study conducted in the less segregated city. The authors offer some practical suggested for future experimental studies.”