BY: Dr. Carmen Callizo Romero & Dr. Shoko Watanabe | July 14, 2025

During Easter, when Carmen was back in her hometown in Murcia, she sent me (Shoko) a photo capturing the procession of the Salzillos—one of the most famous Semana Santa traditions in Spain, where statues carved by the Murcian sculptor Francisco Salzillo Alcaraz, are vibrantly paraded through the city. Seeing the nazarenos all dressed in purple robes carrying the magnificent floats (pasos) projected a parallel image in my head. In Japan, many local summer festivals involve teams of people dressed in colorful uniform carrying lavishly decorated portable mini shrines (omikoshi) throughout their neighborhoods. Very different places, cultures, and religions, but similar emotions, energy, and behavior.

This striking resemblance between the two traditions reminds me of how similar Carmen and I are as researchers despite our different backgrounds. For example, we are both experimental psychologists who have taken “the road less travelled.” Carmen crossed over from philosophy to psychology; I switched from theology to psychology. Consequently, Carmen and I both identify as interdisciplinary scholars because the research topics that excite us the most (such as the cross-culturally recurrent features of religion like cooperating to carry big and heavy sacred objects across town every year) don’t fit neatly within one discipline.

Carmen is a postdoc at the Institute for Culture and Society at the University of Navarra, Spain. Through a grant from the Templeton Foundation, she works with an international team of philosophers, theologians, and psychologists to empirically examine perceptions of divine forgiveness. I’m a postdoc at the University of Birmingham, conducting empirical research for the psychology cross-training for theologians program. In this Templeton-funded project, I also have the opportunity to interact with an international group of theologians.

Carmen and her colleagues recently won an award from the Open Science of Religion Project to conduct an experiment on abstract thinking in cathedrals. I’m currently collaborating with Dr. Carissa Sharp, who also received funding from this open science initiative to conduct an experiment on perceptions of petitionary prayer. Both studies will be published as registered reports in the International Journal for the Psychology of Religion and Archive for the Psychology of Religion, respectively.

As you can see from the non-exhaustive examples above, Carmen and I are like (research) twins separated at birth – we have nearly identical research interests, goals, and career trajectories with only incidental differences. OK…that’s an exaggeration. Nonetheless, because we share a lot in common, it’s no surprise that we also deal with similar challenges, particularly when conducting psychology of religion research. In this blog, we reflect on some of these shared challenges and discuss how Open Science Practices can support collaborative work at the intersection of psychology, theology, and philosophy.

What is Open Science?

Just as different cultures find surprisingly similar ways to celebrate shared values, researchers from different contexts and methods can also find ways to move forward together. This is where Open Science Practices enter the picture, not just as technical improvements, but as tools that can actively strengthen collaboration across diverse fields and worldviews. By “Open Science Practices,” we refer to a growing set of procedures designed to increase the transparency, rigor, and accessibility of research, including preregistration, open data, open materials, and collaborative authorship workflows. These tools (popularized as a way to address the replication crisis) are widely recognized for their crucial role in improving the quality and credibility of research outputs. Less recognized, however, is the additional strength of these transparent practices in facilitating interdisciplinary collaboration, especially when conducting studies about religion–a perennial topic that attracts the attention of scholars across disciplines and cultures.

Notably, Open Science doesn’t have to be an all-or-nothing commitment. We’ve found it helpful to approach it like a buffet. Some practices can be simple, such as sharing stimuli, questionnaires, or conceptual glossaries in public repositories (such as OSF) to enhance accessibility across disciplines or even beyond academia. Other practices require more planning, such as preregistering methodological decisions before data collection or publishing anonymized datasets alongside clearly documented analysis scripts. In collaborative projects, we’ve seen the value of using shared, comment-enabled drafts or analysis protocols. Many of these options can be implemented gradually without needing to follow a rigid “rule book.” For us, this is the true spirit of Open Science: it’s not about checking boxes or conforming to a single technical standard to fulfill discipline-specific norms; it’s more about working with clarity and transparency.

Of course, Open Science doesn’t magically solve all problems, and it has its limitations. Yet, we find this framework particularly valuable in Psychology of Religion and Spirituality research, involving interdisciplinary and cross-cultural studies. What we hope to share here are honest reflections from our experiences in the field, what has helped, and why we think it's worth the effort.

Cross-Cultural Collaborative Research: Expectation vs. Reality

When researchers are interdisciplinary by training and also try to carry out cross-cultural studies, they often find that reality doesn’t quite match their expectations. For example, coming from a background in philosophy and religious studies, I (Carmen) entered psychology PhD, imagining I would write papers with space for deep reflections on time, spirituality, and the meaning of existence. But I quickly discovered that empirical psychology had a very different language. Articles focused on clear hypotheses, quantifiable results, and tightly framed discussions. What I wanted to say had to fit into just a few lines. It was like a culture shock…but also a valuable lesson. I realized that interdisciplinarity doesn’t mean mixing discourses but learning to translate questions and concepts between different worlds.

I finally managed to turn philosophical intuitions and cultural questions into experimental designs. But then came a second challenge: international collaboration. My dissertation was part of a project involving 11 collaborators across countries as diverse as Bosnia and Herzegovina, China, Morocco, and Türkiye. It was the perfect setting to empirically explore a culturally sensitive question in different religious contexts with an interdisciplinary team of experts…what could possibly go wrong?

Our team put a lot of effort into planning the first study. For example, we sought input from bilingual and local cultural experts to promote the same interpretationof the questionnaires in different contexts. But complications quickly emerged. For instance, our participants in Bosnia and Herzegovina reported surprisingly low levels of religiosity, which immediately caught our attention. The pattern was so unexpected that I ended up visiting there to run an extra study to figure out what was happening. But as soon as I arrived, I discovered the real cause: the religiosity questionnaire in Bosnia and Herzegovina used a scale from 0 to 4 when all the other countries used 1 to 5. Because this detail hadn’t been shared with the rest of the team, our initial data made it look like participants from Bosnia and Herzegovina were much less religious than they actually were! This simple error is one example of how small issues can have big consequences, potentially leading to misleading conclusions. These mistakes can be easily caught when you’re working in the same lab in person, but when that’s not possible, you need other safeguards to prevent small errors before they become big problems.

That’s when I began to see the value of Open Science practices. Systematically sharing materials, documenting decisions, and coordinating survey codes across countries helped our team catch any discrepancies and made collaboration smoother, even with external labs! This transparency also allowed our project to expand from 12 to 22 cultural groups in one study.

The biggest test came during the COVID-19 pandemic, when we decided to repeat the original study online with the same groups during lockdown. Thanks to the open workflow that ensured that every member of our research group was on the same page, what could have been chaotic was actually efficient, and we could replicate the study quickly without starting from scratch (see here for more reflections on collaborative cross-cultural work in this study).

At the same time, Open Science wasn’t a magic fix. It didn’t erase tensions between disciplines or fully remove the complexity of coordinating across cultures. But it did provide a flexible, transparent, and accessible framework that held the project together when it mattered. In collaborative teams (from small international projects to big-team science), Open Science helps enhance methodological rigor, diverse perspectives, and shared standards of transparency. For me, it made this international collaboration possible, even across resource gaps and during a global pandemic.

Confirmation Bias: The Double-Edged Sword for Studies of Religion

Carmen mentioned earlier that as researchers, we sometimes find that reality doesn’t match our expectations. When we hypothesize something and conduct a series of experiments and none of the results seem to support our hypothesis, one plausible (and often the most parsimonious) conclusion is that we were wrong. But that’s usually not the first thing that comes to our mind. Instead, we’re motivated to find all the reasons why the experiments didn’t work even though our hypothesis was correct (“it totally would’ve worked if we had used a different measure….maybe the results would be supportive if we performed a different analysis…maybe it wasn’t the right group of participants”). We quickly see these alternative explanations because hindsight is 20/20.

Interestingly, however, we tend to not ask these questions when we find what we expect to find. When we conduct an experiment and the result aligns with our hypothesis, we usually don’t question whether the methods we used were the best way to test our hypothesis. Why? Because humans often fall prey to confirmation bias—the cognitive tendency to search for, interpret, favor, and recall information in a way that confirms our pre-existing beliefs. Even when working with the exact same dataset, researchers can make different analytical decisions, sometimes leading to different outcomes. But scientists are supposed to be objective and unbiased! Yes, but scientists are also people with motivation to publish papers, seek prestige, financial stability, career advancement, etc., and failed experiments (or skepticism about one’s own methodology) don’t help with any of those goals.

Moreover, this problem of confirmation bias is a double-edged sword for researchers who study religion. In addition to the researchers’ own biases that may affect how they report their results, consumers of the research (such as other academics, science journalists, general public) may already have pre-conceived ideas about religion that will either draw them to the findings (if it’s consistent with their beliefs) or lead to outright rejection (if it’s at odds with their beliefs). Decades of attitudes and persuasion research have shown that humans are quite stubborn; we have all kinds of preconceived ideas that are extremely difficult to change. Put concretely, even when we conduct well-theorized, rigorous psychology of religion research, whether or not people accept the empirical findings of our work are sometimes two separate things. To make the reality (empirical findings) fit their expectations (pre-existing beliefs about religion), people might attribute some ideological motives to the researchers, dismiss or ignore the research content, or evaluate methodologies using different standards than what they would use for non-religion related research.

In fact, Dr. Kim Rios’s work has shown that North American social/personality psychologists evaluate research on religion as less intellectually rigorous compared to other subfields of psychology, and they perceive psychologists of religion to be religious by default, more subjective, and less intelligent than researchers in other subfields. If some psychologists feel that way about studies of religion and negatively view their own colleagues who conduct research on religion, we—who are working at the intersection of psychology and theology—really need to be on top of our game.

Open science practices, such as pre-registering studies, will not completely eliminate biases, but it will help reduce their influence by making the researchers accountable. This transparency is valuable for interdisciplinary and cross-cultural projects, where diverse perspectives can help identify blind spots and improve the overall quality of research. This is particularly important for those of us who study religion, as open science practices can add methodological credibility to our research. Even if people aren’t entirely convinced of the research findings (and that’s OK…after all, skepticism is the bedrock of scientific inquiry), we can move forward by displaying our cards openly (rather than hiding them) and inviting critics (rather than shunning them) for open civil discourse. Ultimately, this transparency is what allows science—and especially collaborative interdisciplinary science—to progress.

Two Languages, One Dialogue: Psychology and Theology in Conversation

That psychology is in some tension with theology or philosophy is hardly news to readers of this blog. In an earlier blog, Dr. Keith Dow reminded us that psychology, attempting to distance itself from philosophical and theological roots, has embraced methodological naturalism like a teenage kid embarrassed to be seen with their parents. Well, we believe Open Science can play a meaningful role in promoting a healthy family reunion. For instance, open science practices can foster interdisciplinary dialogue by making a study’s underlying assumptions visible. When researchers preregister a project or share their materials, foundational decisions are no longer implicit. In addition to the analytical details, preregistrations document the specific question(s) that researchers intend to examine and how psychological concepts are operationally defined. This helps scholars from theology or philosophy understand the framework behind a hypothesis and engage critically from the start while also making value-laden terms—like “guilt,” “well-being,” or “salvation”—explicit, inviting reflection on the assumptions shaping the research.

Open science practices can also encourage more hands-on collaboration with philosophers and theologians, for example, by involving them in study design, inviting their feedback on questionnaires, and drawing on sources from their traditions. These contributions are transparently acknowledged. Perhaps most importantly, Open Science pushes us to clarify what we aren’t capturing—what lies beyond the measurable—without dismissing its importance.

These potential benefits aren’t just abstract ideas. For example, in my (Carmen’s) current research on perceptions of Divine Forgiveness, we collaborate with a diverse international team of theologians, philosophers, and psychologists. One of the first and most crucial steps was clarifying what the project was actually studying: not the ontological nature of divine forgiveness, but how people experience, seek, and interpret it. Even when these experiences are rooted in spiritual or sacramental practices, they can still be studied empirically if we understand the theological framework that gives them meaning. That’s where interdisciplinary collaboration was essential. By discussing how we define key concepts across disciplines, we make our research more accessible to scholars from other fields. Relatedly, open science can also make findings and materials more accessible to non-academic audiences. For example, religious leaders, spiritual directors, and other members of faith communities, who may be deeply interested in what our studies reveal, often lack access to paywalled journals. With open access publication or by providing plain-language summary reports through other platforms, researchers can share the fruits of academic labor with the general public, bridging the gap between theory and practice.

Importantly, the goal isn’t to erase the differences between psychology, theology, and philosophy, but to articulate them with clarity and honesty. Or, as Dr. Ela Łazarewicz-Wyrzykowska writes, “The key is not to eliminate the tension by choosing one over the other, but to navigate it thoughtfully to maximise the advantages of both.” Open Science doesn’t eliminate epistemological tensions among disciplines, but it creates the conditions to address them without silencing them. Dialogue doesn’t dilute theoretical frameworks; it makes them more visible, discussable, and generative (for further reflections on this point from the Cross-Training Fellows, see Watanabe et al., 2025 ^[1]). And in this way, many of us find we can be theologians or philosophers at heart and psychologists in practice: not as a contradiction, but as part of a shared vocation toward a more thoughtful and humble science.

Concluding Thoughts: Collaboration (not Competition) as the Game Changer

In a world where scientific progress occurs incrementally, where an individual researcher’s work is invariably one tiny piece of a bigger puzzle, collaborative and cross-disciplinary research sounds like an efficient approach in theory. In reality, however, coordinating among multiple teams with different methodological approaches and lab cultures, in addition to other socio-cultural differences, poses unique challenges particularly for research involving religion. In this blog, we have argued that Open Science Practices can help mitigate some of these challenges. Specifically, providing transparent protocols can help make large-scale collaborations run smoothly while ensuring that studies remain reproducible within and across research teams. This also leads to enhanced credibility by showing that methodology (and not our personal ideologies) is guiding the research. Moreover, by encouraging co-authorship, balanced dialogue, and accessibility of research outputs, Open Science provides a structure where each discipline brings its strengths without fear of being overtaken or dismissed. Opening data also means opening up the conversation.

Finally, we’d like to conclude this blog with a hopeful message for those who might still be undecided—not just about incorporating open science practices into their research but those who are dealing with uncertainties of interdisciplinary or collaborative research.

The academia is a competitive place. Individually, we compete for admissions, jobs, fellowships, grants, awards, recognitions, titles, etc. Systemically, disciplines compete for resources, students, grant funding, prestige, etc. This kind of environment can make us feel lonely because of the pressure to over-perform, be original, and find “novel” effects to be competitive in the job market or for tenure. Similarly, because each discipline is trying to protect its space in academia, blurred boundaries become threatening. One valuable lesson I (Shoko) have learned through the cross-training program is that with collaborative interdisciplinary research, we don’t have to be in competition anymore to see which discipline is “better.” In this sense, interdisciplinary researchers are like MMA fighters.

MMA (mixed-martial arts) is a relatively new sport that began with one question: Which combat style is the most effective? If a wrestler fights a boxer, who wins? Let’s put them in a cage, make them fight for real, and determine this! So at first, the fighters were trained in one basic style (e.g., karate) and fought someone else trained in a different style (e.g., Brazilian jiu-jitsu), but as the sport evolved, the athletes started synthesizing different techniques from various martial arts (they did real cross-training!). Today, MMA is a multi-billion-dollar industry that has carved out its own space as the fastest growing sport in the world, and probably, no modern MMA fighter limits themselves to just one fighting style. I think psychology and theology (and other related disciplines) can stop feeling like we’re in a cage fight. For this reason, I’m thankful for the cross-training program for creating a safe, open, and fun space with mutual trust, where both disciplines can meet and learn from each other. The program embraces the spirit of open science, facilitating scholars from different disciplinary backgrounds with distinct methodologies to accomplish a shared goal, just like the people joining together to carry the pasos or omikoshi.

^[1] Watanabe, S., Kašparová, P., Lazarewicz-Wyrzykowska, E., Perez, J., Tanton, T., & Waite, H. (2025). Voices in the (interdisciplinary) wilderness: Reflections from a psychology cross-training project for theologians. Reviews in Science, Religion and Theology.

We are grateful to Dr. Jordan LaBouff and Dr. Keith Dow for reading over and providing feedback to an earlier version of this blog post.