Despite a new surge of interest, fertility awareness methods (FAMs) remain a critically understudied method of family planning. For women looking for side effect- and hormone-free methods of family planning, the CDC is often the first port of call for information on evidence-based approaches to understanding fertility. When stating the efficacy of FAMs, the CDC gives a wide failure rate range of 2-23%, citing a 2018 meta-analysis by Urritia et al [1].
The Urritia study is a rigorous, detailed analysis. Collectively, FAMs do indeed report a failure rate of 2-23%. Yet this range obscures key differences between methods, apps, and training protocols. These are important differences that women and couples need to know about to be able to make an informed decision about which FAM aligns with their lifestyle and family planning goals, which is a deeply personal and individual choice.
Here we unpack some of the Urritia et al. study results, with the aim of helping women choose the right methods for themselves at the right time of their lives.
What is a meta-analysis?
Urrutia et al. is a meta-analysis, a type of study that aggregates many studies on a particular topic. Ideally, scientific studies will be completely reproducible. Researchers should be able to duplicate the study results solely from following the documentation in the paper. Researchers rarely attain this level of rigor, however. Therefore, statisticians conduct meta-analyses to mitigate the “reproducibility crisis” [2]. The issues that make reproducibility difficult are especially pronounced when studying something as nuanced as FAMs.
Meta-analyses do a great job surveying the field and highlighting holes in the research. But it’s important to recognize the limits of statistics when it comes to applying research to individual decisions. The Urrutia study cited by the CDC gives a range of values for select subpopulations and select methods. This reflects the tremendous variety in the study populations and protocols.
The resulting efficacy rates are helpful guides, but users should not assume they reflect individual experiences with FAMs. Practitioners should draw conclusions from the study carefully and with caveats.
What studies and methods were included in the Urrutia meta-analysis?
The Urrutia meta-analysis considered peer-reviewed articles on all methods of fertility awareness with sufficient documentation. Subsequent analysis categorized them as: Calendar-based methods, Mucus-only methods, Basal Body Temperature (BBT) methods, SymptoThermal methods, and SymptoHormonal methods. The following chart (courtesy of Cassie Moriarty) summarizes the rates for each method. Again, it is important to remember that the Urrutia study gives the rates from a wide range of scientific articles and studies.
Note that FAMs, as with any other method of family planning, are assessed using two rates, the perfect use (efficacy) and typical use rate (effectiveness) of a method. In Urrutia et al., perfect use was similar with Sensiplan (a SymptoThermal method) and Marquette (a SymptoHormonal method), with studies reporting less than one unintended pregnancy rate per year among 100 women. The other methods reported rates generally less than five unintended pregnancies per 100-women years.
What populations did they study?
Urrutia carefully defines inclusion and exclusion criteria for their analysis. Inclusion/exclusion criteria is detailed information about the population in question. Social science doesn’t have set inclusion/exclusion criteria, so rigorous studies document this to be transparent about their results.
Urrutia focused on menstruating and recently pregnant women trying to avoid pregnancy. The group also included a handful of studies on perimenopausal and postpartum women. Studying FAMs for these women remains a critical area for future research, as confirmed by Manhart and Fehring 2018 [3].
There was no limitation on the age of the study for inclusion in the Urrutia meta analysis. While this is inclusive, it means that some methods studied may now be out-of-date, or may not be as commonly used now as they once were.
Urrutia also considered all geographic locations for the study. Again, this means that the wide range of numbers is meant to reflect the experiences of women across the globe. For example, the women in the Sensiplan study were much more likely to be unmarried, without children, and highly educated. It is not a particular leap to suppose that their FAM journey looks rather different than that of married women with multiple children in the developing world.
What is the quality of the studies included in Urrutia?
Statistically, the most rigorous way to assess efficacy is to conduct a randomized controlled trial. This would mean randomly assigning cohorts of women to a particular FAM, and some to a control group. However, this is logistically challenging at best, and unethical at worst. A second key obstacle to rigorous FAM studies is that women are free to leave the study at any time. One of the studies in the meta-analysis reported that over 70% of participants left the study. Without additional information, drawing conclusions from studies with high attrition rates is inadvisable.
While the Urrutia study is the most current survey of the field, perhaps one of the most confusing aspects of the study is that in a methodical review of the body of literature on FAMs, none of the studies were judged to be of high quality. This is largely due to the difficulties in calculating rates for appropriate subgroups.
Social scientists have not agreed on a ranking to define study quality, and setting such a set of criteria is often contentious. Urrutia et al determined that to achieve a rating of “high,” studies must make an explicit attempt to exclude cycles without sexual activity. Contraceptive medication and devices do not necessarily calculate failure rates after excluding cycles without intercourse. This standard for FAMs avoids inflating data with low failure rates due to cycles without intercourse so users can be confident in their choice of method.* However, individuals using FAMs may elect to avoid intercourse during a particular cycle as part of their use of the FAMs. Therefore, this discounting of cycles without intercourse can (unfairly) disadvantage FAMs compared with other methods of family planning that do not exclude these cycles.
A rigorous data collection process would also systematically assess barrier method and withdrawal use during the fertile times in individuals using FAMs. However, very few studies met this criteria. And ideally, new users of the method would be analyzed separately, as practice of a method improves effectiveness. Understanding the effect of sociodemographic characteristics on the efficacy of FAMs is another open question identified by the study.
What are the key takeaways from the study for women using a FAM?
Despite its limits (and the difficulties of studying fertility awareness in general), the Urrutia analysis confirms that FAMs work well. While the CDC uses Urrutia to report a wide range of efficacy values, a closer investigation reveals that many modern FAMs can be highly effective for particular subpopulations and particular methods. Much more research is needed to determine how best to support every kind of woman who wants to use a FAM during any phase of life.
An interactive visual of the key findings in the Urrutia study can be found here. Helpful takeaways for women deliberating about the various fertility awareness methods are summarized in the interactive chart. It gives the basic definition of each method plus important qualitative aspects of each method. Certain FAMs require a certified teacher or offer assistance to women trying to conceive. These qualitative elements are important considerations for women and couples deciding which FAM to use. The chart also differentiates between algorithm-based methods and user-based methods. While the black-and-white output of the algorithms may appeal to some users, others may find user-determined methods more intuitive.
What are the limitations of this study?
The lack of robust, evidenced-based studies and medical provider knowledge on FAMs is wildly disproportionate to the importance of these methods for so many women. The research and data collection process is certainly complex—but not impossible—and women deserve higher quality studies to make informed decisions about their fertility. Recent advances like fertility apps represent a useful data collection resource, but more studies across the whole spectrum of FAMs are needed to accurately represent women’s experiences.
Furthermore, having a child is an intimate decision prompted by deep desires and strings of decisions that don’t reduce into neat columns of zeros and ones. But it is precisely the individual desire (or intention) to plan for or prevent pregnancy that determines the effectiveness of a FAM. Assessing FAMs using the same metrics used for assessing contraceptives cannot assess the holistic impact of FAMs on women’s health.
As many women and couples consider which FAM to use, information about efficacy and effectiveness rates for different methods can be difficult to filter. The Urrutia analysis cited by the CDC in their 2-23% failure rate collates this information and fills this information gap for practitioners. Weighing this information alongside user preferences can be a helpful guide for users to find the FAM best suited to their lifestyle and family planning goals.
*A previous article at Natural Womanhood reported that Urrutia excluded a study on the SymptoThermal method reporting a perfect and typical use rate of 1% and 2% respectively. This is incorrect, as this study, Frank Hermann 2007, was actually included in the Urrutia metanalysis, although it was referred to as Sensiplan [4].
Note: Corrections to calculation errors in the original version of the Urrutia study can be found, here.
References:
1. Peragallo Urrutia, Rachel MD, MS; Polis, Chelsea B. PhD; Jensen, Elizabeth T. PhD; Greene, Margaret E. PhD; Kennedy, Emily MA; Stanford, Joseph B. MD, MSPH Effectiveness of Fertility Awareness–Based Methods for Pregnancy Prevention, Obstetrics & Gynecology: September 2018 – Volume 132 – Issue 3 – p 591-604. doi: 10.1097/AOG.0000000000002784
2. Baker, Monya. “Reproducibility crisis.” Nature 533.26 (2016): 353-66.
3. Manhart, Michael D, and Richard J Fehring. “The State of the Science of Natural Family Planning Fifty Years after Humane Vitae: A Report from NFP Scientists’ Meeting Held at the US Conference of Catholic Bishops, April 4, 2018.” The Linacre quarterly vol. 85,4 (2018): 339-347. doi:10.1177/0024363918809699
4. G. Freundl, E. Godehardt, P.A. Kern, P. Frank‐Herrmann, H.J. Koubenec, Ch. Gnoth, Estimated maximum failure rates of cycle monitors using daily conception probabilities in the menstrual cycle, Human Reproduction, Volume 18, Issue 12, December 2003, Pages 2628–2633, https://doi.org/10.1093/humrep/deg488
Additional Reading:
Journal article describes how effectiveness of FAMs is often misrepresented
This Chart “Perfectly” Displays Perfect vs. Typical Use Effectiveness Rate for Birth Control Methods
CDC Changes Effectiveness Rating on Fertility Awareness Methods
The Ultimate Guide to the Effectiveness of Evidence-Based FAMs
Who tells the truth about the effectiveness of fertility awareness methods?
