This Title Was Not Generated by AI
This podcast was not generated by AI. But it may have been influenced by it. As all industries consider how to use artificial intelligence in their work, Walker Evans, Susan Post, and I discuss how it relates to journalism, disclosure, and transparency. We cover Columbus Underground’s updated policy on the matter, how content generation should and should not be used in a newsroom, what AI thinks about the Columbus food scene, and a lively debate on the ethical challenges of how this new tool will affect our lives.
Shownotes
Read the transcript
Welcome to the Confluence Cast, presented by Columbus Underground. We are a weekly Columbus-centric podcast focusing on the civics, lifestyle, entertainment, and people of our city. I'm your host, Tim Fulton. This week: this podcast was not generated by AI, but it may have been influenced by it. As all industries consider how to use artificial intelligence in their work, Walker Evans, Susan Post, and I discuss how it relates to journalism, disclosure, and transparency. We cover Columbus Underground's updated policy on the matter, how content generation should and should not be used in a newsroom, what AI thinks about the Columbus food scene, and a lively debate on the ethical challenges of how this new tool will affect our lives. You can get more information on what we discussed today in the show notes for this episode at theconfluencecast.com. Enjoy the episode. Sitting down here with Walker Evans, the co-founder of Columbus Underground, and the associate editor of Columbus Underground and Metropreneur, Susan Post. How are you guys? Good. How are you doing? I'm good. You guys have been having a lot of conversations about artificial intelligence and how it affects journalism. And so today, a little bit of an experimental episode, and basically a discussion on AI and those things in journalism: how it may affect Columbus, what are the proper uses for it? So Walker, why don't you kick us off with what those discussions have been like? Well, you know, we've had some writers, like David Staley, who has written about AI in the past for his Next series on CU. We've been hiring some new freelance writers, and one of them asked us in an email, "Hey, what's your AI policy?" And we're like, "We don't have one. I guess we should probably have one." Basically saying, don't use it. We're paying you to do a job. We don't want you to just turn around and ask ChatGPT to do the job for you. So we spent the last couple of months kind of talking about that, and putting some stuff together and kind of getting it out there.
It is a public-facing policy at this point, basically. Yes, you can assure readers and, frankly, advertisers that this content is written by humans. Yes. Yeah. It's at the bottom, like in the footer of the website, on every page. So we're on a site, there's a little link to the AI policy. And so it's on there, but basically just saying we don't use it to write anything. No journalism, no editorial, you know, no images are generated by AI. Our sales team might use it to craft, like, emails, marketing language, kind of thing, which isn't a public-facing thing anyway, right? And some of the tools we use, like Gmail, might have AI capabilities. Photoshop now has AI capabilities. So some of the tools that everyone uses are going to have AI, talking to Siri on your phone, using machine learning, right? So we can't say that we don't ever use it, ever at all, period, because the truth is, like, we all are using it, whether we know it or not. And what about, like, consciously using things like Grammarly, that is using a bit of predictive, like, "Hey, I think you're trying to write this. This seems informational. And you have expressed an opinion." Well, that's kind of the interesting thing about these conversations. Machine learning isn't new, right? So autocorrect is not a new feature. But of course Grammarly needs to say AI on it, because it's like the hottest marketing trend. Just say everything is AI. You turned us on to a great tool: Otter, which is a great one. Everything has an "-ly," and otter.ai is their website, which is ridiculous. But it's an audio transcription. So I will go to a meeting and I'll record the whole thing, and then just feed that audio file in and get a written version of it. Which isn't a new thing. Transcription services have been around for years and years and years. Like, a computer will do it for you. Well, and that's the slippery slope, right? They say they use AI. So it's like, okay, well, I guess now that's an AI thing.
Well, I will say I've used it for — first of all, full disclosure, that is what transcribes the transcripts for the podcasts on your website, which, you know, there has been at least one email where someone said, like, "What a terrible transcription." And we're like, this is actually more of an SEO sort of strategy. Also, it doesn't appear on the Columbus Underground website. It appears on my website. But it has also gotten really good at — I give it the full edited podcast conversation and it comes back and says, "Here's the summary of everything that you said." And I have pulled from it to sort of put together the intro to the podcast. Basically, most people will be able to see the formula. I start off with a statement or question. I talk about who I interview, I list three things we talk about, and then I say, "Enjoy the interview." In the past, like, year or so, some have been pulled from that sort of predictive, like, here's the high-level stuff you talked about. So, yeah, please don't fire me. But yeah, on your website, you know, you clearly label it as a transcript. Like, you know, you're setting that expectation, you're being transparent. So I think there's a difference between that, and then, you know, using it to write a news article. Yeah, yeah. And for us, you know, if I sit down with someone, or we have a 30-minute conversation about something, I want to go back and pull quotes of things that he said to insert into my story. And I don't want to go back and listen to the whole thing and, like, type — and people talk really fast sometimes. So it's a matter of, like, pausing, typing, going back, rewinding, listening, typing, pause, rewind. What did they say? Oh, you know. And instead, just having a tool do that for you, and then you're able to copy-paste it. You're able to control-F first, you're able to go in and be like, "I know they said something like this. So let's dig it up and find it." It's a tool, at the end of the day. Yes, we shouldn't be using it — I don't suggest anyone, like, use it for their homework or anything. But it is a good tool to use in certain contexts, right?
But that's what I'm saying. Like, that audio transcription existed before the AI craze that is going on right now. Do we need a disclaimer on every news article that says, "We used autocorrect"? I typed a word and I misspelled it and autocorrect caught it for me. So therefore AI assisted in the writing of this article? Like, that's a little over the top, I think. We have seen, even in the Columbus sort of journalism sphere, publications using AI to generate articles. Specifically, the Columbus Dispatch was called out for — I think it was a local sports game, like a high school sports game. They took the scores, they took the runners, and, like, it seemed templatized more than anything else, but it got things wrong. And so they immediately were like, "Oh, we don't do that anymore." And just to be fair to the journalists here in Columbus, who do great work at the Dispatch, I think these kinds of decisions to use these kinds of tools are being made by Gannett, the parent company. I don't think that local sports writers are like, "Hey, let's fire the staff and replace it with this, like, garbage robot stuff," right? So the public responded pretty negatively to that. Well, at the end of the day, it's a trust thing. Like, can I trust what you're reading? I mean, God forbid — just based on our last episode — God forbid somebody took the transcript from a school board meeting, transcribed it, put that real rough transcription (because to be clear, Otter is not perfect), and then made sure it was assigning the right roles for who talked, fed that into ChatGPT and said, "Write me an article summarizing the things that happened in this school board meeting. Please include two pull quotes from different school board members about significant things at this meeting." And copy and paste, and nobody really copy-edits it, or nobody listens to it, or reads the entire transcript. Again, a trust issue. It's super important to have actual, like, boots on the ground.
Yeah. I guess I just wonder, like, if you're going to give it that rigorous validation and testing and making sure that, like, this has been fact-checked, are you really saving any time yourself? Right? Yeah, that's all I was thinking while you're saying that. Like, you have to tell it so many prompts to get it anywhere close to something reasonable that it's like, I can just do it myself that way. But so I have — I have played with it a lot, to the extent that I've given it a full transcript, or like an Otter AI summary, and say, "Write me a podcast introduction for this." It's always ridiculous and over the top. But that is another thing that I'm able to be like, "Oh, that's an interesting idea. I'll rephrase that." That's a little bit different. Like, podcasting production is not a breaking news article.
True, right? Absolutely. Yeah. But to that end, you can create, like, little basically self-formed GPTs within their model and say, like, "Hey, you are a writer for The Confluence Cast. It's a Columbus-centric podcast. Please write an introduction for this," and basically save that prompt, and then just put in the thing each time. So, like, it's getting smarter and smarter. I, frankly, personally am excited about how far we can go and not be damaging, right? Sure. But isn't it human nature to run things off a cliff? I have a higher — maybe — vote of confidence for humanity than you. I hope we don't go that direction. I'm just saying that bad actors will always, yes, do things bad with it. To Susan's point about breaking news, I mean, it's not something that can get things right. Like, it will never be a replacement for that, right? You know what I mean? And that's a lot of what we do, is, like, of-the-minute, you know, sort of stuff. Additionally, you know, it can't really give you an opinion. It could fake it, but it's not real. So we can't replace Miriam, our restaurant reviewer, with ChatGPT. It's not going to tell us how a hamburger tastes, right?
Or it's going to pull someone else's view and tell what that person thought it tastes like. And then you're plagiarizing other people's stuff. That's the problem, too. A lot of these things don't really fully tell you where it's getting its information from, right? So — I just wrote an opinion piece, and I kind of, just as examples, fed it some prompts to show. And one of them was, like, kind of pointing out that it's great if you're doing stuff very generically. Yes. Or something you don't know how to do, like write a business plan. I'm talking specifically about journalism. If you want a listicle, it will generate a listicle. And so I said, "Give me a listicle: five hamburger restaurants in Columbus, Ohio." Okay, it spat back a very generic listicle. And the Thurman Burger was right up — of course — number one. Number five, though —
Was Cray Eatery and Drinkery? Cray is not around anymore. A restaurant that closed eight years ago, in — 2016. And it said it was on High Street. But it didn't know about Cray. Like, Cray is somewhere in the bot.
That, too, because it's closed? So if you want to, like, generate bad information, I don't know. To me, it's like, to what end are you doing stuff? You know, if I want to generate 1,000, like, SEO-centric listicles for Columbus Underground to spit them out there, I'm having a robot write material for robots to read, right? Like, are we actually doing journalism anymore at this point? True? Absolutely. Take that to sort of the internet, right? And, like, how do you make money on the internet? If it's just the internet, it's advertising. And how do you get eyes on something without paying for eyes to get on something? It's through SEO. And, like — shout out to Brad Keefe, we were just texting about this this morning. He was like, SEO is writing for humans and robots. And people actually get the human part wrong a lot. And ChatGPT is also going to get the human part wrong, but it's likely going to get the robot part pretty right. But there's also, like, a quality to it. Is your reaction to that — like, you can generate a lot of garbage and get a lot of garbage eyeballs. But you're not building a lasting audience. You're not building trust. You're not getting return visitors. You're just sort of scraping the bottom of the internet barrel at that point for couch change.
Yep. I mean, yeah, it's a bench billboard. Yeah. Again, absolutely. Well, I think, you know, we have a lot of discussions about kind of what our audience expects of us. And, like, you know, they will call it out if something is crappy. And they should, you know. And so I think it's really important to us to always make sure we're putting quality work out there, and not just, like, "Go try this burger. It's great." You know, like, that's not what we do. And that's not what people want to see from us. Well, speaking of high-quality stuff that our readers are asking for — I've typed into ChatGPT in front of me: "Give us five topics for discussion in a podcast about Columbus, Ohio." Can we see what it says? Well, hold on. I'm putting it into ChatGPT — the pro version. Yes. Give us — what is it, five —
"Give us five topics for discussion in a podcast about Columbus, Ohio." And to back up, too: we did a forum with the Columbus Metropolitan Club, which I'm on the programming committee for. And I said, "You know, we should do a topic on AI." They hadn't really done anything — this is last summer. And kind of the gimmicky, fun sort of thing was, we had a human moderator, two human panelists, and then the third panelist was ChatGPT. It invented the questions and it was reading it out loud in a voice, and, like, everyone was sort of like, "Oh, this is kind of cool. I want to see what it has to say." And it was a nice — it was a one-on-one sort of discussion. But when I was demonstrating to the board, because a lot of the folks there have, like, heard of it, but they never really messed around with it — I'm like, "Well, this is what we can do." And I typed in a prompt like this and said, "Give me several topics for potential Columbus Metropolitan Club forum ideas," right? And it spat back like eight of them. And they all worked. But they were very generic. It's like, you could talk about the growth in the healthcare sector, urban development and infill that's happening with this, there's changes in, like, workforce retention. And it's like, well, these are all kind of forums that they already do. They already hit all these topics. But I am going to push back on you here. Like, that's a prompt problem, right? Okay. If you hone in a little bit more, and basically — first of all, tell it who it is. You are a programmer for this forum. You want to fill a variety of specific things, you know, because I imagine programmatically, they're like, "Okay, every fiscal quarter, we've got to hit an economic issue. We've got to hit our healthcare issue." And given that, right, you might get more specific. So let's check our GPT. Okay, so yours — to be clear, it says, "Give us five talk topics for discussion in a podcast for" — I said "about Columbus" — okay, I just want to be exactly the same. So — Comma, Ohio, period. "About." Okay, enter.
I'm very curious. I've never personally used ChatGPT. Finished. Okay, you want — You want me to just — Go with your first one, and I'll tell you what my first one was.
Number one? What says — "Certainly, here are five engaging topics that could be discussed in the podcast." I always read the responses in, like, game show voice, yelling, like a commercial, right? Yeah. Like radio. Mine also says "five engaging topics," but here we go. Number one: exploring Columbus's diverse neighborhoods. Okay, my first one: the evolution of Columbus. Okay, what's your second one?
Second one is the evolution of the Ohio State — Mine is the Columbus culinary scene. Okay. It's interesting that it's not the same, right? Yeah. Number three is Columbus's role in Ohio's political landscape. Yours is better. Mine is, like, arts and — yours is arts and culture, and —
Saving money. Yeah. Number four is the Columbus arts and music scene. Very similar. Innovations and business in — Five for me is innovations and startups in Columbus. And I have community and lifestyle. Like, mine just literally named sections of the newspaper.
Yeah, pretty much. I think culinary was the only one that was not in this list. I had that. So broad. It is. Yeah, it is, right. Yeah. Like, you can ask it to narrow and ask it to focus and ask it to get more specific and stuff like that, but —
Well, and so I'm copying and pasting and writing, right? "Five podcast topics around the evolution of Columbus." And it's giving me: "From Franklinton to the capital: the founding and early years of Columbus." And it's writing about that. "Columbus in conflict, colon: the Civil War and its impact on the city." Three: "The industrial boom and urban expansion, colon: Columbus in the late 19th and early 20th century." Sounds like WOSU. They do — Four: "Cultural melting pot, colon: the influence of immigration and diversity in shaping Columbus." And five: "Modern Columbus, colon: urban renewal, tech innovation, and the city's future." Like, they do sound like — that's what I mean by, if you keep going down the path, right. Well, and this is maybe my ignorance question, because I know the data set stops at, what, January 2022, something like that? So it's about two years behind. If you ask it, like, "What are the best new restaurants in Columbus to visit?" What does it do? What does it think, at —
At least mine would come back and say, "I do not know. My data stops here. And here is what I know about before that." It's pretty explicit about that. Yeah. When I asked, like, "Write a news article about the car crash that just happened outside," it's like, "I'm sorry to hear about the car that just crashed. I don't have information on that," and blah, blah. Well, but I do. I've listened to a couple of podcasts about it from a technical standpoint, and from a, like, "how should we think about this" standpoint. And the thought I have is, like, this is like a calculator. Like, this is a phone. This is a television set. This is the internet. This is going to make things more efficient. And yes, if you trick it to say racist things, it's going to say racist things. But first of all, why are you doing that? Like, why are you trying to get it to do terrible things? Like, I can make a Play-Doh penis all I want, but why would I do that? Like, that's not what it's for. I'm saying — well, you just made a face. Tell me. Yeah, sorry. I'll get to that in one sec. But I guess, you know, because we said "five engaging topics for a podcast about Columbus, Ohio," this is the same issue that I had with the CMC stuff, was like, "Give us forums on this." Yeah, replace Columbus, Ohio with Pittsburgh, Pennsylvania, because of the healthcare industry, the big college, diversity in the workforce — like, it's just kind of paint-by-numbers. And the reason that they asked me to be on that panel is to come with ideas, right? And this is, like, "Okay, these are, like, C-plus ideas." Like, I'm there to provide something a little bit more than that, right? I don't know. Anyway, now that I asked the question of "What are the best new restaurants in Columbus?" — number one, Alberta's Pizza Kitchen?
Not familiar. I don't know if that's — okay, apologies to Alberta. Sorry, Alberta. Starting a new GPT when you didn't know? I — I did. Yeah, I know that you're supposed to, if you're wildly off topic. But some of these I think are newer, so it's probably pulling information —
Where? "Do you want" — one South High Street, according to my Googling. 541 South High — 541 South High. Brewery District. Is it a ghost kitchen?
I don't know. So — hey, I just asked GPT and it started by saying, "My last update is April 2023, but here you go." Oh, you know what it did? It said my last update was April 2023, so here are some ways to go find — you know, go blog, Yelp, word of mouth. Oh. It's in — Veritas? Classics? Oh. All right. Well, sorry. But Piazza Paolino, which is newer —
Which — that opened in December. And so, go. You got quite a GPT over there, I guess. Yeah. Why are you paying for the premium? Hai is in here. Cobra's in here. But it's like a really weird list, because, like, Birdie's Grill at the fairway? I don't know. That's up in Dublin, maybe. Okay. Bowl of boba. I have no idea where that is.
We need to be checking this for story ideas. Also, Apartment B, which is, like, the speakeasy thing behind Bodega, which is a bar, not a restaurant. But Thunderwing Brewing, our friends' brewery — that does not have food. Then where is it pulling this information from? Well, you ask it that, and it's like, "Can't really say. I have been fed all this information coming from everywhere." Which, like, if you asked a journalist, like, "Cite your sources," and they're just like, "I looked it up online," you'd be like, "You're fired."
And there was just — this morning, as of recording — something came across my feed that said most major news sources have put in the ad to do-not-call this for generative AI. Oh, yeah. Which is — have you done that? I'm just curious. Um, I know Justin, our main web developer — we were talking about it. I can't remember if he implemented it or not. That we wanted to opt out of stuff, though. Okay. Because it's stealing. Well, there's also some interesting kind of, like, think pieces that I've seen about, like, if this is based on what's out there, but what's out there in the near future is stuff generated by this, you get this, like, slowly it becomes — it degrades. It's like a fax of a fax, a copy of a copy of a fax, and all of a sudden nothing is correct anymore. That has — that started, like, what, six months after GPT came out, where — and this is me following, like, tech podcasts — where they're like, "Yeah, they have to refresh the model every now and again to basically do that." And then there's a whole subset of, like, copyright issues to be addressed and all that stuff. But I guess I hope that there will be a curated environment where this is a healthy, productive tool. But yeah, I do not want to be told by what I think is a person, meaning a news organization, something that's regurgitated because of their $20-a-month subscription to ChatGPT. Yeah. So I think one of the coolest applications of this that I've heard about is the ability to take a large library of audio, of voice content — someone speaking, like yourself — and then be able to transcribe, or not transcribe, but translate into other languages. So being able to — podcast specifically — be able to listen to Confluence Cast in French, and it's our voices. It's — c'est Français, right? Because you have the voice data to be able to do that. It's like, we don't need to know how to speak French. You could easily, you know, take a podcast recorded in one language and translate it into 200 different languages, like that, and open up a lot of accessibility. The downside is you're going to trick a lot of old grandmas into thinking that Joe Biden said something that he didn't actually say. Yes.
Yeah. Well, and there's, like — it's easy to use here, too, right? But there is a scary use case of, like, that didn't happen. Will Smith didn't eat that spaghetti. And there's just a lot, but so real. But this does go back to the entire, like, media literacy argument that we've been having for years with misinformation and all that stuff. This suite of technology has simply sped it up. True. So, yeah. I think a lot about how, you know, when the printing press became a thing, it kind of allowed people to mass produce literature. You know, so the first newspapers and, you know, that sort of stuff was able to be made, but you had to afford the printing press and the paper and the material. So, like, not everyone in the world — Yeah, it started the newspaper. Well, it's the argument, like, "Don't get in a fight with somebody who buys ink by the barrel." Yes. So ChatGPT launched and it was on everyone's phone in the world overnight. So everyone had access to use this tool, and it just speeds up not only the ability to make it better, but also the ability to become a bad actor very quickly.
Yes. And I think — yeah, the whole stuff with Sam Altman — and I didn't quite — I got very tired of following it for that weekend. I have to hope that the folks who are in control, for lack of a better word, are consistently monitoring, consistently, like, "How do we make this better and safer?" So that, like — You hope, but do you really think that it's going to be like that, or is money going to win out in the end? As a — hello, cynicism. Well, no, I get that, but I would argue that money winning out in the end means that people are going to be like, "We can't generate a tool that hurts people," right? Like, we can't — Google's — "believe us, we, the board of directors, we —" The Facebook — what kind of damage has that already done to people?
I mean, every big corporation in the world does things that do damage to people. I understand what you're saying. But I guess the argument I would make is, when that does happen, and then we become aware of it — and I will admit that, like, I should probably be more cynical here, but the market corrects itself. The market does sort of say, like, "Hey, this is unethical," or, frankly, we elect people that should be regulating the space, you know. And if they don't, God bless globalism, the EU will. We all just got to move to — Well, no, no. But the choices that they are making — as somebody who works in the tech field, the choices that they're making — like, I have to adhere to cookie policies in California if I want to sell to people in California, right? And in the EU if I want to sell to people in the EU. And, like, those liberal democracies — and I use small-l there — will make things — there will be somebody checking it. If you're not watching this on video, Walker's just, like, doing this "I don't know" face.
Yeah. I mean, I think as long as money is the bottom line, ethics will always take a backseat. Okay. Yeah, fair. I'm not saying that as a cynic. I'm just saying, like, you look at any major corporation, and, like, they're only, like, three steps removed from, like, slave labor on the other side of the world, whether it's shoes or — It will —
Precious metals. There's a whole lot of, you know, a cottage industry of people who actually go and check these things, right? And, yes, they are not infallible, but there are people. I want clean food, I want shoes that were made ethically. And I realize that that's not, like, a thing. But at least the labor that I don't like is actually better than the work available to those laborers in the place where that labor is done, if that makes sense. Like, rising tide. I am an optimist about our brand new calculator. Each in the middle. You be the optimist, I'll be the pessimist, and just hope it all shakes out okay for everyone, right? Thank you guys for your time today.
Yeah, this is fun. Thank you. Thank you for listening to the Confluence Cast, presented by Columbus Underground. Again, you can get more information on what we discussed today in the show notes for this episode at theconfluencecast.com. Please rate, subscribe, and share this episode of The Confluence Cast with your friends, family, contacts, enemies, your favorite prompt engineer. If you're interested in sponsoring the Confluence Cast, get in touch with us. We can be reached by email at info@theconfluencecast.com. Our theme music was composed by Benji Robinson. Our producer is Philip Cogley. I'm your host, Tim Fulton. Have a great week.
Transcript5,120 words
Tim Fulton Welcome to the Confluence Cast, presented by Columbus Underground. We are a weekly Columbus-centric podcast focusing on the civics, lifestyle, entertainment, and people of our city. I'm your host, Tim Fulton. This week: this podcast was not generated by AI, but it may have been influenced by it. As all industries consider how to use artificial intelligence in their work, Walker Evans, Susan Post, and I discuss how it relates to journalism, disclosure, and transparency. We cover Columbus Underground's updated policy on the matter, how content generation should and should not be used in a newsroom, what AI thinks about the Columbus food scene, and a lively debate on the ethical challenges of how this new tool will affect our lives. You can get more information on what we discussed today in the show notes for this episode at theconfluencecast.com. Enjoy the episode. Sitting down here with Walker Evans, the co-founder of Columbus Underground, and the associate editor of Columbus Underground and Metropreneur, Susan Post. How are you guys?
Walker Evans Good. How are you doing?
Tim Fulton I'm good. You guys have been having a lot of conversations about artificial intelligence and how it affects journalism. And so today, a little bit of an experimental episode, and basically a discussion on AI and those things in journalism: how it may affect Columbus, what are the proper uses for it? So Walker, why don't you kick us off with what those discussions have been like?
Walker Evans Well, you know, we've had some writers, like David Staley, who has written about AI in the past for his Next series on CU. We've been hiring some new freelance writers, and one of them asked us in an email, "Hey, what's your AI policy?" And we're like, "We don't have one. I guess we should probably have one." Basically saying, don't use it. We're paying you to do a job. We don't want you to just turn around and ask ChatGPT to do the job for you. So we spent the last couple of months kind of talking about that, and putting some stuff together and kind of getting it out there.
Tim Fulton It is a public-facing policy at this point, basically. Yes, you can assure readers and, frankly, advertisers that this content is written by humans.
Walker Evans Yes. Yeah. It's at the bottom, like in the footer of the website, on every page. So we're on a site, there's a little link to the AI policy. And so it's on there, but basically just saying we don't use it to write anything. No journalism, no editorial, you know, no images are generated by AI. Our sales team might use it to craft, like, emails, marketing language, kind of thing, which isn't a public-facing thing anyway, right? And some of the tools we use, like Gmail, might have AI capabilities. Photoshop now has AI capabilities. So some of the tools that everyone uses are going to have AI, talking to Siri on your phone, using machine learning, right? So we can't say that we don't ever use it, ever at all, period, because the truth is, like, we all are using it, whether we know it or not.
Tim Fulton And what about, like, consciously using things like Grammarly, that is using a bit of predictive, like, "Hey, I think you're trying to write this. This seems informational. And you have expressed an opinion."
Walker Evans Well, that's kind of the interesting thing about these conversations. Machine learning isn't new, right? So autocorrect is not a new feature. But of course Grammarly needs to say AI on it, because it's like the hottest marketing trend. Just say everything is AI. You turned us on to a great tool: Otter, which is a great one. Everything has an "-ly," and otter.ai is their website, which is ridiculous. But it's an audio transcription. So I will go to a meeting and I'll record the whole thing, and then just feed that audio file in and get a written version of it. Which isn't a new thing. Transcription services have been around for years and years and years. Like, a computer will do it for you. Well, and that's the slippery slope, right? They say they use AI. So it's like, okay, well, I guess now that's an AI thing.
Tim Fulton Well, I will say I've used it for — first of all, full disclosure, that is what transcribes the transcripts for the podcasts on your website, which, you know, there has been at least one email where someone said, like, "What a terrible transcription." And we're like, this is actually more of an SEO sort of strategy. Also, it doesn't appear on the Columbus Underground website. It appears on my website. But it has also gotten really good at — I give it the full edited podcast conversation and it comes back and says, "Here's the summary of everything that you said." And I have pulled from it to sort of put together the intro to the podcast. Basically, most people will be able to see the formula. I start off with a statement or question. I talk about who I interview, I list three things we talk about, and then I say, "Enjoy the interview." In the past, like, year or so, some have been pulled from that sort of predictive, like, here's the high-level stuff you talked about. So, yeah, please don't fire me.
Susan Post But yeah, on your website, you know, you clearly label it as a transcript. Like, you know, you're setting that expectation, you're being transparent. So I think there's a difference between that, and then, you know, using it to write a news article.
Walker Evans Yeah, yeah. And for us, you know, if I sit down with someone, or we have a 30-minute conversation about something, I want to go back and pull quotes of things that he said to insert into my story. And I don't want to go back and listen to the whole thing and, like, type — and people talk really fast sometimes. So it's a matter of, like, pausing, typing, going back, rewinding, listening, typing, pause, rewind. What did they say? Oh, you know. And instead, just having a tool do that for you, and then you're able to copy-paste it.
Tim Fulton You're able to control-F first, you're able to go in and be like, "I know they said something like this. So let's dig it up and find it." It's a tool, at the end of the day. Yes, we shouldn't be using it — I don't suggest anyone, like, use it for their homework or anything. But it is a good tool to use in certain contexts, right?
Walker Evans But that's what I'm saying. Like, that audio transcription existed before the AI craze that is going on right now. Do we need a disclaimer on every news article that says, "We used autocorrect"? I typed a word and I misspelled it and autocorrect caught it for me. So therefore AI assisted in the writing of this article? Like, that's a little over the top, I think.
Tim Fulton We have seen, even in the Columbus sort of journalism sphere, publications using AI to generate articles. Specifically, the Columbus Dispatch was called out for — I think it was a local sports game, like a high school sports game. They took the scores, they took the runners, and, like, it seemed templatized more than anything else, but it got things wrong. And so they immediately were like, "Oh, we don't do that anymore."
Walker Evans And just to be fair to the journalists here in Columbus, who do great work at the Dispatch, I think these kinds of decisions to use these kinds of tools are being made by Gannett, the parent company. I don't think that local sports writers are like, "Hey, let's fire the staff and replace it with this, like, garbage robot stuff," right? So the public responded pretty negatively to that.
Tim Fulton Well, at the end of the day, it's a trust thing. Like, can I trust what you're reading? I mean, God forbid — just based on our last episode — God forbid somebody took the transcript from a school board meeting, transcribed it, put that real rough transcription (because to be clear, Otter is not perfect), and then made sure it was assigning the right roles for who talked, fed that into ChatGPT and said, "Write me an article summarizing the things that happened in this school board meeting. Please include two pull quotes from different school board members about significant things at this meeting." And copy and paste, and nobody really copy-edits it, or nobody listens to it, or reads the entire transcript. Again, a trust issue. It's super important to have actual, like, boots on the ground.
Walker Evans Yeah. I guess I just wonder, like, if you're going to give it that rigorous validation and testing and making sure that, like, this has been fact-checked, are you really saving any time yourself?
Susan Post Right? Yeah, that's all I was thinking while you're saying that. Like, you have to tell it so many prompts to get it anywhere close to something reasonable that it's like, I can just do it myself that way.
Tim Fulton But so I have — I have played with it a lot, to the extent that I've given it a full transcript, or like an Otter AI summary, and say, "Write me a podcast introduction for this." It's always ridiculous and over the top. But that is another thing that I'm able to be like, "Oh, that's an interesting idea. I'll rephrase that."
Susan Post That's a little bit different. Like, podcasting production is not a breaking news article.
Tim Fulton True, right? Absolutely. Yeah. But to that end, you can create, like, little basically self-formed GPTs within their model and say, like, "Hey, you are a writer for The Confluence Cast. It's a Columbus-centric podcast. Please write an introduction for this," and basically save that prompt, and then just put in the thing each time. So, like, it's getting smarter and smarter. I, frankly, personally am excited about how far we can go and not be damaging, right?
Walker Evans Sure. But isn't it human nature to run things off a cliff?
Tim Fulton I have a higher — maybe — vote of confidence for humanity than you.
Walker Evans I hope we don't go that direction. I'm just saying that bad actors will always, yes, do things bad with it. To Susan's point about breaking news, I mean, it's not something that can get things right. Like, it will never be a replacement for that, right? You know what I mean? And that's a lot of what we do, is, like, of-the-minute, you know, sort of stuff. Additionally, you know, it can't really give you an opinion. It could fake it, but it's not real. So we can't replace Miriam, our restaurant reviewer, with ChatGPT. It's not going to tell us how a hamburger tastes, right?
Susan Post Or it's going to pull someone else's view and tell what that person thought it tastes like.
Walker Evans And then you're plagiarizing other people's stuff. That's the problem, too. A lot of these things don't really fully tell you where it's getting its information from, right? So — I just wrote an opinion piece, and I kind of, just as examples, fed it some prompts to show. And one of them was, like, kind of pointing out that it's great if you're doing stuff very generically.
Tim Fulton Yes. Or something you don't know how to do, like write a business plan.
Walker Evans I'm talking specifically about journalism. If you want a listicle, it will generate a listicle. And so I said, "Give me a listicle: five hamburger restaurants in Columbus, Ohio." Okay, it spat back a very generic listicle. And the Thurman Burger was right up — of course — number one. Number five, though —
Tim Fulton Was Cray Eatery and Drinkery? Cray is not around anymore.
Susan Post A restaurant that closed eight years ago, in —
Walker Evans 2016. And it said it was on High Street.
Tim Fulton But it didn't know about Cray. Like, Cray is somewhere in the bot.
Susan Post That, too, because it's closed?
Walker Evans So if you want to, like, generate bad information, I don't know. To me, it's like, to what end are you doing stuff? You know, if I want to generate 1,000, like, SEO-centric listicles for Columbus Underground to spit them out there, I'm having a robot write material for robots to read, right? Like, are we actually doing journalism anymore at this point?
Tim Fulton True? Absolutely. Take that to sort of the internet, right? And, like, how do you make money on the internet? If it's just the internet, it's advertising. And how do you get eyes on something without paying for eyes to get on something? It's through SEO. And, like — shout out to Brad Keefe, we were just texting about this this morning. He was like, SEO is writing for humans and robots. And people actually get the human part wrong a lot. And ChatGPT is also going to get the human part wrong, but it's likely going to get the robot part pretty right.
Walker Evans But there's also, like, a quality to it. Is your reaction to that — like, you can generate a lot of garbage and get a lot of garbage eyeballs. But you're not building a lasting audience. You're not building trust. You're not getting return visitors. You're just sort of scraping the bottom of the internet barrel at that point for couch change.
Tim Fulton Yep. I mean, yeah, it's a bench billboard. Yeah. Again, absolutely.
Susan Post Well, I think, you know, we have a lot of discussions about kind of what our audience expects of us. And, like, you know, they will call it out if something is crappy. And they should, you know. And so I think it's really important to us to always make sure we're putting quality work out there, and not just, like, "Go try this burger. It's great." You know, like, that's not what we do. And that's not what people want to see from us.
Walker Evans Well, speaking of high-quality stuff that our readers are asking for — I've typed into ChatGPT in front of me: "Give us five topics for discussion in a podcast about Columbus, Ohio." Can we see what it says?
Tim Fulton Well, hold on. I'm putting it into ChatGPT — the pro version. Yes. Give us — what is it, five —
Walker Evans "Give us five topics for discussion in a podcast about Columbus, Ohio." And to back up, too: we did a forum with the Columbus Metropolitan Club, which I'm on the programming committee for. And I said, "You know, we should do a topic on AI." They hadn't really done anything — this is last summer. And kind of the gimmicky, fun sort of thing was, we had a human moderator, two human panelists, and then the third panelist was ChatGPT. It invented the questions and it was reading it out loud in a voice, and, like, everyone was sort of like, "Oh, this is kind of cool. I want to see what it has to say." And it was a nice — it was a one-on-one sort of discussion. But when I was demonstrating to the board, because a lot of the folks there have, like, heard of it, but they never really messed around with it — I'm like, "Well, this is what we can do." And I typed in a prompt like this and said, "Give me several topics for potential Columbus Metropolitan Club forum ideas," right? And it spat back like eight of them. And they all worked. But they were very generic. It's like, you could talk about the growth in the healthcare sector, urban development and infill that's happening with this, there's changes in, like, workforce retention. And it's like, well, these are all kind of forums that they already do. They already hit all these topics.
Tim Fulton But I am going to push back on you here. Like, that's a prompt problem, right? Okay. If you hone in a little bit more, and basically — first of all, tell it who it is. You are a programmer for this forum. You want to fill a variety of specific things, you know, because I imagine programmatically, they're like, "Okay, every fiscal quarter, we've got to hit an economic issue. We've got to hit our healthcare issue." And given that, right, you might get more specific. So let's check our GPT. Okay, so yours — to be clear, it says, "Give us five talk topics for discussion in a podcast for" — I said "about Columbus" — okay, I just want to be exactly the same. So —
Walker Evans Comma, Ohio, period.
Tim Fulton "About." Okay, enter.
Susan Post I'm very curious. I've never personally used ChatGPT.
Tim Fulton Finished. Okay, you want —
Walker Evans You want me to just —
Tim Fulton Go with your first one, and I'll tell you what my first one was.
Walker Evans Number one? What says — "Certainly, here are five engaging topics that could be discussed in the podcast." I always read the responses in, like, game show voice, yelling, like a commercial, right? Yeah. Like radio.
Tim Fulton Mine also says "five engaging topics," but here we go.
Walker Evans Number one: exploring Columbus's diverse neighborhoods.
Tim Fulton Okay, my first one: the evolution of Columbus. Okay, what's your second one?
Walker Evans Second one is the evolution of the Ohio State —
Tim Fulton Mine is the Columbus culinary scene. Okay. It's interesting that it's not the same, right?
Walker Evans Yeah. Number three is Columbus's role in Ohio's political landscape.
Tim Fulton Yours is better. Mine is, like, arts and — yours is arts and culture, and —
Walker Evans Saving money. Yeah. Number four is the Columbus arts and music scene.
Tim Fulton Very similar. Innovations and business in —
Walker Evans Five for me is innovations and startups in Columbus.
Tim Fulton And I have community and lifestyle. Like, mine just literally named sections of the newspaper.
Walker Evans Yeah, pretty much. I think culinary was the only one that was not in this list. I had that.
Susan Post So broad.
Tim Fulton It is. Yeah, it is, right.
Walker Evans Yeah. Like, you can ask it to narrow and ask it to focus and ask it to get more specific and stuff like that, but —
Tim Fulton Well, and so I'm copying and pasting and writing, right? "Five podcast topics around the evolution of Columbus." And it's giving me: "From Franklinton to the capital: the founding and early years of Columbus." And it's writing about that. "Columbus in conflict, colon: the Civil War and its impact on the city." Three: "The industrial boom and urban expansion, colon: Columbus in the late 19th and early 20th century."
Walker Evans Sounds like WOSU. They do —
Tim Fulton Four: "Cultural melting pot, colon: the influence of immigration and diversity in shaping Columbus." And five: "Modern Columbus, colon: urban renewal, tech innovation, and the city's future." Like, they do sound like — that's what I mean by, if you keep going down the path, right.
Susan Post Well, and this is maybe my ignorance question, because I know the data set stops at, what, January 2022, something like that? So it's about two years behind. If you ask it, like, "What are the best new restaurants in Columbus to visit?" What does it do? What does it think, at —
Tim Fulton At least mine would come back and say, "I do not know. My data stops here. And here is what I know about before that." It's pretty explicit about that.
Walker Evans Yeah. When I asked, like, "Write a news article about the car crash that just happened outside," it's like, "I'm sorry to hear about the car that just crashed. I don't have information on that," and blah, blah.
Tim Fulton Well, but I do. I've listened to a couple of podcasts about it from a technical standpoint, and from a, like, "how should we think about this" standpoint. And the thought I have is, like, this is like a calculator. Like, this is a phone. This is a television set. This is the internet. This is going to make things more efficient. And yes, if you trick it to say racist things, it's going to say racist things. But first of all, why are you doing that? Like, why are you trying to get it to do terrible things? Like, I can make a Play-Doh penis all I want, but why would I do that? Like, that's not what it's for. I'm saying — well, you just made a face. Tell me.
Walker Evans Yeah, sorry. I'll get to that in one sec. But I guess, you know, because we said "five engaging topics for a podcast about Columbus, Ohio," this is the same issue that I had with the CMC stuff, was like, "Give us forums on this." Yeah, replace Columbus, Ohio with Pittsburgh, Pennsylvania, because of the healthcare industry, the big college, diversity in the workforce — like, it's just kind of paint-by-numbers. And the reason that they asked me to be on that panel is to come with ideas, right? And this is, like, "Okay, these are, like, C-plus ideas." Like, I'm there to provide something a little bit more than that, right? I don't know. Anyway, now that I asked the question of "What are the best new restaurants in Columbus?" — number one, Alberta's Pizza Kitchen?
Tim Fulton Not familiar.
Walker Evans I don't know if that's — okay, apologies to Alberta. Sorry, Alberta.
Tim Fulton Starting a new GPT when you didn't know? I —
Walker Evans I did. Yeah, I know that you're supposed to, if you're wildly off topic. But some of these I think are newer, so it's probably pulling information —
Susan Post Where? "Do you want" — one South High Street, according to my Googling. 541 South High —
Walker Evans 541 South High.
Susan Post Brewery District.
Walker Evans Is it a ghost kitchen?
Susan Post I don't know.
Tim Fulton So — hey, I just asked GPT and it started by saying, "My last update is April 2023, but here you go." Oh, you know what it did? It said my last update was April 2023, so here are some ways to go find — you know, go blog, Yelp, word of mouth. Oh.
Susan Post It's in — Veritas? Classics? Oh.
Walker Evans All right. Well, sorry. But Piazza Paolino, which is newer —
Susan Post Which — that opened in December.
Walker Evans And so, go.
Tim Fulton You got quite a GPT over there, I guess.
Walker Evans Yeah. Why are you paying for the premium? Hai is in here. Cobra's in here. But it's like a really weird list, because, like, Birdie's Grill at the fairway? I don't know. That's up in Dublin, maybe. Okay. Bowl of boba. I have no idea where that is.
Susan Post We need to be checking this for story ideas.
Walker Evans Also, Apartment B, which is, like, the speakeasy thing behind Bodega, which is a bar, not a restaurant. But Thunderwing Brewing, our friends' brewery — that does not have food.
Susan Post Then where is it pulling this information from?
Walker Evans Well, you ask it that, and it's like, "Can't really say. I have been fed all this information coming from everywhere." Which, like, if you asked a journalist, like, "Cite your sources," and they're just like, "I looked it up online," you'd be like, "You're fired."
Tim Fulton And there was just — this morning, as of recording — something came across my feed that said most major news sources have put in the ad to do-not-call this for generative AI. Oh, yeah. Which is — have you done that? I'm just curious.
Walker Evans Um, I know Justin, our main web developer — we were talking about it. I can't remember if he implemented it or not. That we wanted to opt out of stuff, though. Okay. Because it's stealing. Well, there's also some interesting kind of, like, think pieces that I've seen about, like, if this is based on what's out there, but what's out there in the near future is stuff generated by this, you get this, like, slowly it becomes — it degrades. It's like a fax of a fax, a copy of a copy of a fax, and all of a sudden nothing is correct anymore.
Tim Fulton That has — that started, like, what, six months after GPT came out, where — and this is me following, like, tech podcasts — where they're like, "Yeah, they have to refresh the model every now and again to basically do that." And then there's a whole subset of, like, copyright issues to be addressed and all that stuff. But I guess I hope that there will be a curated environment where this is a healthy, productive tool. But yeah, I do not want to be told by what I think is a person, meaning a news organization, something that's regurgitated because of their $20-a-month subscription to ChatGPT.
Walker Evans Yeah. So I think one of the coolest applications of this that I've heard about is the ability to take a large library of audio, of voice content — someone speaking, like yourself — and then be able to transcribe, or not transcribe, but translate into other languages. So being able to — podcast specifically — be able to listen to Confluence Cast in French, and it's our voices. It's — c'est Français, right? Because you have the voice data to be able to do that. It's like, we don't need to know how to speak French. You could easily, you know, take a podcast recorded in one language and translate it into 200 different languages, like that, and open up a lot of accessibility. The downside is you're going to trick a lot of old grandmas into thinking that Joe Biden said something that he didn't actually say. Yes.
Tim Fulton Yeah. Well, and there's, like — it's easy to use here, too, right? But there is a scary use case of, like, that didn't happen. Will Smith didn't eat that spaghetti. And there's just a lot, but so real. But this does go back to the entire, like, media literacy argument that we've been having for years with misinformation and all that stuff. This suite of technology has simply sped it up. True. So, yeah.
Walker Evans I think a lot about how, you know, when the printing press became a thing, it kind of allowed people to mass produce literature. You know, so the first newspapers and, you know, that sort of stuff was able to be made, but you had to afford the printing press and the paper and the material. So, like, not everyone in the world —
Tim Fulton Yeah, it started the newspaper. Well, it's the argument, like, "Don't get in a fight with somebody who buys ink by the barrel." Yes.
Walker Evans So ChatGPT launched and it was on everyone's phone in the world overnight. So everyone had access to use this tool, and it just speeds up not only the ability to make it better, but also the ability to become a bad actor very quickly.
Tim Fulton Yes. And I think — yeah, the whole stuff with Sam Altman — and I didn't quite — I got very tired of following it for that weekend. I have to hope that the folks who are in control, for lack of a better word, are consistently monitoring, consistently, like, "How do we make this better and safer?" So that, like —
Susan Post You hope, but do you really think that it's going to be like that, or is money going to win out in the end?
Tim Fulton As a — hello, cynicism. Well, no, I get that, but I would argue that money winning out in the end means that people are going to be like, "We can't generate a tool that hurts people," right? Like, we can't — Google's — "believe us, we, the board of directors, we —"
Susan Post The Facebook — what kind of damage has that already done to people?
Walker Evans I mean, every big corporation in the world does things that do damage to people.
Tim Fulton I understand what you're saying. But I guess the argument I would make is, when that does happen, and then we become aware of it — and I will admit that, like, I should probably be more cynical here, but the market corrects itself. The market does sort of say, like, "Hey, this is unethical," or, frankly, we elect people that should be regulating the space, you know. And if they don't, God bless globalism, the EU will.
Walker Evans We all just got to move to —
Tim Fulton Well, no, no. But the choices that they are making — as somebody who works in the tech field, the choices that they're making — like, I have to adhere to cookie policies in California if I want to sell to people in California, right? And in the EU if I want to sell to people in the EU. And, like, those liberal democracies — and I use small-l there — will make things — there will be somebody checking it. If you're not watching this on video, Walker's just, like, doing this "I don't know" face.
Walker Evans Yeah. I mean, I think as long as money is the bottom line, ethics will always take a backseat.
Tim Fulton Okay. Yeah, fair.
Walker Evans I'm not saying that as a cynic. I'm just saying, like, you look at any major corporation, and, like, they're only, like, three steps removed from, like, slave labor on the other side of the world, whether it's shoes or —
Tim Fulton It will —
Walker Evans Precious metals.
Tim Fulton There's a whole lot of, you know, a cottage industry of people who actually go and check these things, right? And, yes, they are not infallible, but there are people. I want clean food, I want shoes that were made ethically. And I realize that that's not, like, a thing. But at least the labor that I don't like is actually better than the work available to those laborers in the place where that labor is done, if that makes sense. Like, rising tide. I am an optimist about our brand new calculator.
Walker Evans Each in the middle. You be the optimist, I'll be the pessimist, and just hope it all shakes out okay for everyone, right?
Tim Fulton Thank you guys for your time today.
Walker Evans Yeah, this is fun. Thank you.
Tim Fulton Thank you for listening to the Confluence Cast, presented by Columbus Underground. Again, you can get more information on what we discussed today in the show notes for this episode at theconfluencecast.com. Please rate, subscribe, and share this episode of The Confluence Cast with your friends, family, contacts, enemies, your favorite prompt engineer. If you're interested in sponsoring the Confluence Cast, get in touch with us. We can be reached by email at info@theconfluencecast.com. Our theme music was composed by Benji Robinson. Our producer is Philip Cogley. I'm your host, Tim Fulton. Have a great week.