Buddhist Catholic Contemplative Evangelical General Christian Hindu Jewish Latter-day Saint Muslim New Visions Progressive Christian

Beliefs

Topics

Columnists

Resource Library

Research Tools

Video

Podcasts

Thought Readers

Faith In Media Tracker

Newsletters

Ad-Free & Subscriber Log-In Subscription Support

Follow Patheos

Home
Progressive Christian
Technology and Religion
Can a Chatbot Reason?

Can a Chatbot Reason?

Last updated on: July 24, 2025 at 7:50 pm by James F. McGrath

0 Comments

Can a Chatbot Reason? 2025-07-24T19:50:08-04:00 James F. McGrath

LOGO Patheos Explore the world's faith through different perspectives on religion and spirituality! Patheos has the views of the prevalent religions and spiritualities of the world.

I had the opportunity to have a really interesting conversation on Reddit about LLMs and reasoning. The person who wrote the original post expressed surprise and frustration that ChatGPT says things that are incorrect and then apologizes. I wrote, “It imitates speech. It has no mechanism for recognizing facts or information. They just happen to be woven into the speech patterns it is imitating. That you find what it does surprising makes me suspect you may not be aware of what this technology is at its most fundamental level.” That led to a discussion with someone who disagreed with me. They shared a post to an “article” (it is better referred to as a blog post or press release) from Anthropic, claiming to get at how an LLM reasons. My initial response was to write, ‘It is not about regurgitation. It follows patterns in language that match the kinds of responses appropriate to the input speech. Those patterns will of course reflect the way semantic concepts are networked in human speech, because that was the training data. When a speech-imitating chatbot is asked to “show its work” it generates the kind of output that would be appropriate if it were showing its work.’ They were still unsatisfied with my perspective, and I got the impression that they have some expertise in this area. What I wrote next is this:

“Forgive me if I get something wrong in what follows. I’ve consulted extensively and even co-written things with a computer science colleague, but that isn’t my own field. However, I cannot see that this sort of swapping indicates something happening other than what I indicated. In the training data, Austin will appear in relation to Texas in similar patterns to Sacramento and California, will it not? And so when one term is changed, one would expect the other to be changed. This only indicates that it has learned the game of language and appropriate moves and countermoves in the same way that a neural network does with other games such as Go or chess.”

They were inspired to write a detailed response, which you can read on Reddit. Here is what I wrote:

‘You refer to a what is happening “on a conceptual level” and I would love clarification about what that means in your understanding of this technology. As for me, I have no doubt that by imitating what happens in human speech, an LLM-powered chatbot can do things that mirror the reasoning that humans express through such language. My own impression is that this is no different than the fact that a chess-playing AI will seem to “want to win” because its moves mirror those of a human who wants to win. Yet there is no justification for treating that as indicating the AI has desires that correspond to what human game-players experience.

Where I think we may disagree is on precisely a philosophical matter. You appear to believe that what we as humans do would appear to a silicon-based life form the same as what LLMs do appears to us. I am inclined to think that there are crucial aspects of our reasoning that depend on our subjective experience and consciousness, but that will be challenging to prove in a way that would persuade a skeptic. Let us revisit the analogy that I think we both agree is helpful. An LLM learns and plays language in a manner akin to how it learns the game chess and plays it. The key question that may or may not be a difference of opinion between us is whether the subjective experience of strategizing, competing, and feeling satisfaction or frustration are inherent in what it means to play chess. Turning to language, an LLM-powered chatbot can always without fail make winning moves if it is judged in terms of what it was trained to do, namely play the game of human language. If we suddenly change the criteria and make the evaluation be about accuracy of information, then suddenly its ability to “win” drops exponentially. My explanation is that it was not trained to, and has not evolved as a surprising emergent capacity, the ability to identify facts and information. It plays the game of human language, and the reason it wins more often than not when judged in terms of information is (in my view) that there is so much information woven into human speech patterns that playing the game of language also provides information a significant amount of the time.

The key question, if I am right about where we agree and disagree, is whether managing the landscape of information is something that may have emerged from neural networks trained to imitate language, would be something an AI system could do if specifically trained to do so (assuming we can figure out how to do this thing that humans struggle to), or requires a self-awareness of the sort that humans have, even if we are not consistently successful when we try. I do not deny that imitating language at times succeeds in imitating the underlying cognitive processes and reasoning that we then express in language. What I do deny (although not adamantly, and I am willing to be persuaded otherwise) is that imitating language will consistently accomplish what human reasoning does, much less that doing so deserves to be considered essentially the same thing because it is supposedly functionally equivalent.

In the interest of full disclosure, I write this as a professor who is chair of a department of philosophy and religious studies and who has just co-authored a book with a colleague in computer science on how to teach the humanities in the era of generative AI. I am also the author of some essays on AI and religion and a novelette about AI. So I am someone who has given this topic a lot of thought and who approaches it through the lens of philosophy and ethics, but who is not a computer scientist and so I recognize my need to rely on the expertise of others when engaging in interdisciplinary conversations of this sort. Thank you for this opportunity to learn and reflect further!’

That is where the conversation had reached so far. You’ve hopefully read other things I’ve written here about chatbots, and may also have seen my recent article in Interpretation about using chatbots for sermon preparation, a topic that was also in the news recently.

ChatGPT Can Interpret Speaking in Tongues?

John’s Adoptionist Christology? Limits of AI Lit Reviews

Artificial Intelligence and Plagiarism

Of related interest:

AI and Higher Ed impending collapse

Trump targets “woke AI”

When Your Girlfriend Is an Algorithm (Part 2)

Will AI outsmart human intelligence? – with ‘Godfather of AI’ Geoffrey Hinton

Ex-Google CEO Eric Schmidt: What Artificial Superintelligence Will Look Like

Responsible Use of AI in Bible Study

Filling in the “Anti-Woke” Void

AI deceptions and politicians

Technology and Religion

artificial intelligence

chatbot

ChatGPT

LLM

Previous Post

July 14, 2025

Using AI to Highlight and Challenge Stereotypes

Next Post

September 8, 2025 SNTS 2025 Regensburg

Recent Comments

0 | Leave a Comment

"V. good.For me, the gold standard for this sort of thing is still Wikihistory by ..."

Scurra

The Time-Traveler’s Apprentice (A Short Story)

"The one who says "no one comes to the Father but through me" is the ..."

James F. McGrath

Attack on Religious Freedom in Indiana

"Until I read, "As a Christian, it is possible but not at all necessary to ..."

Sam Vimes

Attack on Religious Freedom in Indiana

"Hey James, regarding your penultimate paragraph, I struggle with the same issue. I do think ..."

Jeff Cate

The Fourth Synoptic Gospel

Browse Our Archives

Select a Category 1 Corinthians 1 John 1 Peter 1 Samuel 1 Thessalonians 1 Timothy 2 Corinthians 2 Kings 2 Peter 2 Samuel 2 Thessalonians 2 Timothy academia Acts of the Apostles afterlife Akhaten Alcatraz American Academy of Religion Amos Anglican Arabic Aramaic archaeology art artificial intelligence astronomy atheism atonement Babylon 5 Baptists Battlestar Galactica Bible Biblical Studies Carnival biblioblogging Black Mirror Blade Runner Book of Job book review Buddhism Call for Papers China Christianity Christmas Christology Chronicles Coptic creation Daniel Deuteronomy Doctor Who Dune Easter Ecclesiastes economics education Eldad Keynan Enoch Seminar Ephesians eschatology Esther Evangelicals evolution Exodus Expanse Ezekiel Ezra fantasy Fringe fundamentalism Galatians games Ge'ez genealogy Genesis Gnosticism Gospel of Jesus' Wife Gospel of John Gospel of Judas Gospel of Luke Gospel of Mark Gospel of Matthew Gospel of Peter Gospel of Thomas Greek Harry Potter Hebrew Hebrews Hinduism historical Jesus history Hitchhiker's Guide to the Galaxy Holidays horror Hosea humor incarnation India inerrancy information literacy inspirational Intelligent Design Ireland Isaiah Islam Israel Italy James Jeremiah John the Baptist Jonah Joshua Judaism Jude Judges Judith Lamentations languages Latin Leviticus libraries literature Lord of the Rings LOST Mandaeans Mandaic memes Methodism Micah monotheism morality Mormons movies music mythicism NASSCAL Nehemiah new perspective New Testament news Numbers Orthodox paganism panentheism pantheism Patheos Paul and his letters Pentecostals Philippians Philippians 2:6-11 philosophy physics Planet of the Apes podcast politics Progressive Christianity Protestantism Proverbs Psalms pseudoscience racism religion Revelation Roman Catholicism Romania Romans Ruth scams science science fiction Secret Gospel of Mark Society of Biblical Literature Song of Songs Star Trek Star Wars Stargate Swahili Syriac Taoism technology Terminator The Handmaid's Tale The Hobbit The Matrix theodicy theology theory Timeless Titus tomb Torchwood travel TRON Uncategorized United Kingdom Westworld X-Files young-earth creationism Zechariah

Select a Month October 2026 September 2026 August 2026 July 2026 June 2026 May 2026 April 2026 March 2026 February 2026 January 2026 December 2025 November 2025 October 2025 September 2025 July 2025 June 2025 May 2025 April 2025 March 2025 February 2025 January 2025 December 2024 November 2024 October 2024 September 2024 August 2024 July 2024 June 2024 May 2024 April 2024 March 2024 February 2024 January 2024 December 2023 November 2023 October 2023 September 2023 August 2023 July 2023 May 2023 April 2023 March 2023 February 2023 January 2023 December 2022 November 2022 October 2022 September 2022 August 2022 July 2022 June 2022 May 2022 April 2022 March 2022 February 2022 January 2022 December 2021 November 2021 October 2021 September 2021 August 2021 July 2021 June 2021 May 2021 April 2021 March 2021 February 2021 January 2021 December 2020 November 2020 October 2020 September 2020 August 2020 July 2020 June 2020 May 2020 April 2020 March 2020 February 2020 January 2020 December 2019 November 2019 October 2019 September 2019 August 2019 July 2019 June 2019 May 2019 April 2019 March 2019 February 2019 January 2019 December 2018 November 2018 October 2018 September 2018 August 2018 July 2018 June 2018 May 2018 April 2018 March 2018 February 2018 January 2018 December 2017 November 2017 October 2017 September 2017 August 2017 July 2017 June 2017 May 2017 April 2017 March 2017 February 2017 January 2017 December 2016 November 2016 October 2016 September 2016 August 2016 July 2016 June 2016 May 2016 April 2016 March 2016 February 2016 January 2016 December 2015 November 2015 October 2015 September 2015 August 2015 July 2015 June 2015 May 2015 April 2015 March 2015 February 2015 January 2015 December 2014 November 2014 October 2014 September 2014 August 2014 July 2014 June 2014 May 2014 April 2014 March 2014 February 2014 January 2014 December 2013 November 2013 October 2013 September 2013 August 2013 July 2013 June 2013 May 2013 April 2013 March 2013 February 2013 January 2013 December 2012 November 2012 October 2012 September 2012 August 2012 July 2012 June 2012 May 2012 April 2012 March 2012 February 2012 January 2012 December 2011 November 2011 October 2011 September 2011 August 2011 July 2011 June 2011 May 2011 April 2011 March 2011 February 2011 January 2011 December 2010 November 2010 October 2010 September 2010 August 2010 July 2010 June 2010 May 2010 April 2010 March 2010 February 2010 January 2010 December 2009 November 2009 October 2009 September 2009 August 2009 July 2009 June 2009 May 2009 April 2009 March 2009 February 2009 January 2009 December 2008 November 2008 October 2008 September 2008 August 2008 July 2008 June 2008 May 2008 April 2008 March 2008 February 2008 January 2008 December 2007 November 2007 October 2007 September 2007 August 2007 July 2007 June 2007 February 2007 January 2001

TAKE THE

Religious Wisdom Quiz